Predicting likelihood and site of metastasis from patient records

ABSTRACT

Systems and methods are provided for predicting metastasis of a cancer in a subject. A plurality of data elements for the subject&#39;s cancer is obtained, including sequence features comprising relative abundance values for gene expression in a cancer biopsy of the subject, optional personal characteristics about the subject, and optional clinical features related to the stage, histopathological grade, diagnosis, symptom, comorbidity, and/or treatment of the cancer in the subject, and/or a temporal element associated therewith. One or more models are applied to the plurality of data elements, determining one or more indications of whether the cancer will metastasize. A clinical report comprising the one or more indications is generated.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/373,451, filed on Jul. 12, 2021, entitled “Predicting Likelihood andSite of Metastasis from Patient Records,” which is a continuation ofU.S. patent application Ser. No. 17/227,120, filed on Apr. 9, 2021,entitled “Predicting Likelihood and Site of Metastasis from PatientRecords,” which claims priority to U.S. Provisional Patent ApplicationNo. 63/007,874, filed on Apr. 9, 2020, entitled “Predicting Likelihoodand Site of Metastasis from Patient Records,” and U.S. ProvisionalPatent Application No. 63/142,051, filed on Jan. 27, 2021, entitled“Predicting Likelihood and Site of Metastasis from Patient Records,” thedisclosures of which are incorporated by reference herein, in theirentireties, for all purposes.

FIELD OF THE INVENTION

The present disclosure relates to computer-implemented methods andsystems for predicting a likelihood that a patient tumor metastasizes toanother organ in the patient based on computational analysis ofpatient's medical records.

BACKGROUND

Precision oncology is the practice of tailoring cancer therapy to theunique genomic, epigenetic, and/or transcriptomic profile of anindividual's cancer. Personalized cancer treatment builds uponconventional therapeutic regimens used to treat cancer based only on thegross classification of the cancer, e.g., treating all breast cancerpatients with a first therapy and all lung cancer patients with a secondtherapy. This field was borne out of many observations that differentpatients diagnosed with the same type of cancer, e.g., breast cancer,responded very differently to common treatment regimens. Over time,researchers have identified genomic, epigenetic, and transcriptomicmarkers that improve predictions as to how an individual cancer willrespond to a particular treatment modality.

There is growing evidence that cancer patients who receive therapyguided by their genetics have better outcomes. For example, studies haveshown that targeted therapies result in significantly improvedprogression-free cancer survival. See, e.g., Radovich M. et al.,Oncotarget, 7(35):56491-500 (2016). Similarly, reports from the IMPACTtrial—a large (n=1307) retrospective analysis of consecutive,prospectively molecularly profiled patients with advanced cancer whoparticipated in a large, personalized medicine trial—indicate thatpatients receiving targeted therapies matched to their tumor biology hada response rate of 16.2%, as opposed to a response rate of 5.2% forpatients receiving non-matched therapy. Tsimberidou A M et al., ASCO2018, Abstract LBA2553 (2018).

In fact, therapy targeted to specific genomic alterations is already thestandard of care in several tumor types, e.g., as suggested in theNational Comprehensive Cancer Network (NCCN) guidelines for melanoma,colorectal cancer, and non-small cell lung cancer. In practice,implementation of these targeted therapies requires determining thestatus of the diagnostic marker in each eligible cancer patient. Whilethis can be accomplished for the few, well known mutations associatedwith treatment recommendations in the NCCN guidelines using individualassays or small next generation sequencing (NGS) panels, the growingnumber of actionable genomic alterations and increasing complexity ofdiagnostic classifiers necessitates a more comprehensive evaluation ofeach patient's cancer genome, epigenome, and/or transcriptome.

For instance, cancer metastasis, which is the spread of a cancer from aprimary site to other sites within the body, is a hallmark of anadvanced cancer stage and is generally associated with poorer clinicaloutcomes. Metastatic cancers are more commonly treated with systemictherapies than are primary cancers without signs of metastasis, whichare commonly treated with local therapy. For instance, a metastaticcancer may be treated with both a local therapy to address a particulartumor (e.g., a tumor at the primary site of cancer and/or a tumor at ametastasis site) and with a systemic adjuvant and/or neoadjuvant therapyto kill cancer cells that have spread to other parts of the body,thereby reducing the probability that further metastatic disease willdevelop. However, systemic treatments are generally more toxic to apatient's body than are local therapies, and are associated with sideeffects such as nausea, fatigue, low white blood cell count,myelosuppression, and nerve damage, which negatively impact a patient'squality of life. Because the ability to metastasize is not an inherentability of all cancers, not all cancer patients will benefit fromadjuvant therapy.

However, predicting which patients will most benefit from adjuvanttherapy is difficult. For instance, Xu W. et al., BMC Medicine, 18:172(2020) conducted an in-depth evaluation and cross-assessment of 34 riskfactors and 12 prediction models for colorectal cancer metastasis andfound convincing evidence for the association of only a single riskfactor (vascular invasion) with colorectal cancer metastasis, and thatassociation was limited to lymph node metastasis in pT1 tumors. As such,many cancer patients are unnecessarily treated with adjuvant therapiesdue in large part to the inability to accurately predict which cancerswill ultimately metastasize and which cancers will not. van 't Veer L.J. et al., Nature, 415(6871):530-36 (2002).

Extracting meaningful medical features from an ever-expanding quantityof health information tabulated for a similarly expanding cohort ofpatients having a multitude of sparsely populated features is adifficult endeavor. Identifying which medical features, from the tens ofthousands of features available in health information, are mostprobative to training and utilizing a prediction engine only compoundsthe difficulty. Features which may be relevant to predictions may onlybe available in a small subset of patients and features which are notrelevant may be available in many patients. What is needed is a systemthat may ingest these impossibly comprehensive scope of available dataacross entire populations of patients to identify features which applyto the largest number of patients and establish a model for predictionof an objective. When there are multiple objectives to choose from, whatis needed is a system which may curate the medical features extractedfrom patient health information to a specific model associated with theprediction of the desired objective. One relevant objective is tocompute the likelihood that a patient's cancer will metastasize to aspecifically identified organ in the patient's body within a definedperiod of time after one or more events, such as next-generationsequencing (NGS) of the patient's tumor.

SUMMARY

Given the above background, improved systems and methods are needed forpredicting the likelihood that a patient's cancer will metastasize(e.g., to a specific tissue and/or within a specific time frame), forexample to improve access to personalized therapies. Advantageously, thepresent disclosure provides solutions to these and other shortcomings inthe art.

One aspect of the present disclosure provides a method for predictingmetastasis of a cancer in a subject, at a computer system having one ormore processors, and memory storing one or more programs for executionby the one or more processors. The method includes obtaining, inelectronic format, a plurality of data elements for the subject'scancer.

The plurality of data elements includes a first set of sequence featurescomprising relative abundance values (e.g., expression values for aplurality of genes) for the expression of a plurality of genes (e.g., atleast 30 genes) in a biopsy of the cancer obtained from the subject. Insome embodiments, the plurality of data elements includes one or morepersonal characteristics about the subject selected from the groupconsisting of age, gender, and race. In some embodiments, the pluralityof data elements further comprises one or more clinical features relatedto the diagnosis or treatment of the cancer in the subject selected fromthe group consisting of a stage of the cancer, a histopathological gradeof the cancer, a therapy administered to the subject, a symptomassociated with cancer or metastasis thereof, and a comorbidity with thecancer. In some embodiments, the plurality of data elements furthercomprises one or more temporal features related to the diagnosis ortreatment of the cancer in the subject selected from the groupconsisting of a first temporal element indicating the duration of timesince a diagnosis for the cancer (e.g., a diagnosis of the cancer, agrading of the cancer, and/or a histopathological grading of thecancer), a second temporal element indicating the duration of time sincean administration of the therapy (e.g., since first administration,since last administration, and/or since completion of a firsttherapeutic regimen), a third temporal element indicating the durationof time since an experience of the symptom (e.g., since first experienceof the symptom and/or since last experience of the symptom), and afourth temporal element indicating the duration of time since anexperience of the comorbidity (e.g., since the beginning of thecomorbidity and/or since the resolution of the comorbidity).

The method includes applying, to the plurality of data elements for thesubject's cancer, one or more models (e.g., predictive and/orclassification models) that are collectively trained to provide arespective one or more indications (e.g., a binary indication, alikelihood and/or a probability) of whether the cancer will metastasizein the subject (e.g., to any tissue site within a respective single timehorizon), thereby predicting whether the cancer will metastasize.

The method further includes generating a clinical report comprising theone or more indications of whether the cancer will metastasize (e.g.,one or more indications of metastasis to any tissue site within arespective single time horizon).

In some embodiments, the method includes applying, to the plurality ofdata elements for the subject's cancer, a set of models (e.g.,predictive and/or classification models) that are collectively trainedto provide, for each respective tissue in a plurality of tissues (e.g.,at different locations within a subject's body), a respective set ofindications of whether the cancer will metastasize to the respectivetissue in the subject (e.g., a binary indication, a likelihood and/or aprobability). The respective set of indications includes a respectiveindication for each respective time horizon in a plurality of timehorizons. Thus, in some such embodiments, the method determines aplurality of indications of whether the cancer will metastasize thatincludes, for each respective tissue in the plurality of tissues, arespective set of indications comprising, for each respective timehorizon in a plurality of time horizons, a respective indication ofwhether the cancer in the subject will metastasize to the respectivetissue within the respective time horizon.

In some embodiments, systems and methods are provided for generating andmodeling predictions of patient objectives (e.g., generating, training,and applying models for predicting an objective based on featuresassociated with a patient). The model(s) can be selected based onamount, type, and other properties of information available for apatient. The systems and methods provide techniques for computationalprocessing of information in patient records (e.g., varioussemi-structured and unstructured data) to convert the information into aformat suitable for use in the predictive models. Thus, in someembodiments, interactions are identified in a patient record, and, forevery identified interaction, a prediction of an objective may becalculated. The prediction can relate to, for example, a likelihood ofmetastasis within a certain time from the respective interaction pointand/or specific location(s) of the metastasis. The predictions areidentified using a model that can be selected from a plurality of modelsbased on the available patient information.

INCORPORATION BY REFERENCE

All publications, patents, and patent applications mentioned in thisspecification are herein incorporated by reference in their entiretiesto the same extent as if each individual publication, patent, or patentapplication was specifically and individually indicated to beincorporated by reference.

BRIEF DESCRIPTION OF THE DRAWINGS

The figures described below depict various aspects of the system andmethods disclosed herein. It should be understood that each figuredepicts an example of aspects of the present systems and methods.

FIG. 1 is a block diagram illustrating a system for generatingpredictions of an objective from a plurality of patient features, inaccordance with some embodiments of the present disclosure.

FIG. 2 is a block diagram illustrating a system for performingselection, alteration, and calculation of additional features from thepatient features, in accordance with some embodiments of the presentdisclosure.

FIG. 3 is a schematic illustration of an example of a system forselecting a feature set for generating prior features and forwardfeatures based on a target/objective pair, in accordance with someembodiments of the present disclosure.

FIG. 4 is a schematic illustration of an example of a system forselecting a feature set for generating prior features based onpredicting the likelihood that a patient's cancer will metastasize to aspecifically identified organ in the patient's body within a definedperiod of time, in accordance with some embodiments of the presentdisclosure.

FIG. 5 is a schematic illustration of a system for selecting a featureset for generating prior features based from predicting the likelihoodthat a patient's cancer will metastasize to a specifically identifiedorgan in the patient's body within a defined period of time, inaccordance with some embodiments of the present disclosure.

FIG. 6 is a flowchart illustrating a method for generating priorfeatures and providing the prior features to a model for predicting thelikelihood that a patient's cancer will metastasize to a specificallyidentified organ in the patient's body within a defined period of time,in accordance with some embodiments of the present disclosure.

FIG. 7 is an illustration of an example of a patient timeline havingevents determining prior and forward features, in accordance with someembodiments of the present disclosure.

FIG. 8 is a flowchart illustrating a method for performing analytics inconjunction with application of a model for predicting site-specificmetastasis in a patient, in accordance with some embodiments of thepresent disclosure.

FIG. 9 illustrates an example of elements of a webform for viewingsite-specific predictions of metastasis in a patient, in accordance withsome embodiments of the present disclosure.

FIG. 10 illustrates an example of elements of a webform for viewingsite-specific predictions of metastasis in a cohort of patients, inaccordance with some embodiments of the present disclosure.

FIG. 11 illustrates an example of elements of a webform for viewingfeature rankings of site-specific predictions of metastasis in a cohortof patients, in accordance with some embodiments of the presentdisclosure.

FIG. 12 illustrates an example of elements of a webform for viewingfeature importance rankings of site-specific predictions of metastasisin a cohort of patients, in accordance with some embodiments of thepresent disclosure.

FIG. 13 is illustrates an example of aggregate measures of performanceacross classification thresholds of input datasets according to anobjective of predicting the likelihood that a patient's cancer willmetastasize to a specifically identified organ in the patient's bodywithin a defined period of time, in accordance with some embodiments ofthe present disclosure.

FIG. 14 is a block diagram of an example of a system in which someembodiments of the invention can be implemented.

FIG. 15 illustrates a block diagram of an example computing device, inaccordance with some embodiments of the present disclosure.

FIG. 16 provides a flow chart of processes and features for predictingmetastasis of a cancer in a subject, in which optional blocks areindicated with dashed boxes, in accordance with some embodiments of thepresent disclosure.

FIG. 17 illustrates a schematic of one or more classification models forpredicting metastasis of a cancer in a subject, in which optionalfeatures are indicated with dashed boxes, in accordance with someembodiments of the present disclosure.

FIG. 18 illustrates a schematic of a classification model for predictingmetastasis of a cancer in a subject, in accordance with some embodimentsof the present disclosure.

FIG. 19A illustrates survival curves of a survival model showingpredicted metastasis-free survival for high-risk patients (1902) andlow-risk patients (1904), in accordance with some embodiments of thepresent disclosure.

FIG. 19B illustrates a histogram of model scores output from an entiretraining cohort, in accordance with some embodiments of the presentdisclosure.

FIG. 20 illustrates survival curves of a survival model showingpredicted metastasis-free survival for high-risk patients (2002) andlow-risk patients (2004), in accordance with some embodiments of thepresent disclosure.

FIG. 21 illustrates survival curves of a survival model showingpredicted metastasis-free survival for high-risk patients (2102) andlow-risk patients (2104), in accordance with some embodiments of thepresent disclosure.

FIG. 22 illustrates survival curves of a survival model showingpredicted metastasis-free survival for high-risk patients (2202) andlow-risk patients (2204), in accordance with some embodiments of thepresent disclosure.

FIG. 23 illustrates an example report generated based on the analysis ofpatient data using a predictive model of metastasis, in accordance withsome embodiments of the disclosure.

DETAILED DESCRIPTION

Among other aspects, the present disclosure provides methods and systemsfor predicting whether a cancer will metastasize based on one or morecharacteristics of the subject. In some embodiments, the methods andsystems described herein evaluate at least a portion of thetranscriptome from a sample of the cancer tissue when predicting whetherthe cancer will metastasize. In some embodiments, the methods andsystems further evaluate one or more personal characteristics of thesubject, one or more clinical features of the cancer, one or morepathological features of the cancer, or one or more additional nucleicacid-based features of the cancer when predicting whether the cancerwill metastasize. Also provided herein are methods for training modelsfor predicting whether a cancer will metastasize.

Generating and Modeling Predictions of Patient Objectives.

FIG. 1 illustrates an embodiment of a computer-implemented system 100for generating and modeling predictions of patient objectives.Predictions may be generated from patient information represented byfeature modules 110 implemented by the system architecture 100. Thesystem 100 can be a content server (also referred to as a predictionengine), which is hardware or a combination of both hardware andsoftware. A user, such as a health care provider or patient, is givenremote access through the GUI to view, update, and analyze informationabout a patient's medical condition using the user's own local device(e.g., a personal computer or wireless handheld device). A user caninteract with the system to instruct it to generate electronic records,update the electronic records, and perform other actions. The contentserver is configured to receive various information in different formatsand it converts the information into the standardized format that issuitable for processing by modules operation on or in conjunction withthe content server. Thus, information acquired from patients' electronicmedical records (EMR), unstructured text, genetic sequencing, imaging,and various other information can be converted into features that areused for training a plurality of machine-learning models.

The information acquired, processed, and generated by the content server100 is stored on one or more of the network-based storage devices. Theuser can interact with the content server to access the informationstored in the network-based storage devices, and the content server canreceive user-supplied information, apply the one or more models storedin the network-based storage to the information, and to provide, in anelectronic form, results of the model application to the user on agraphical user interface of the user device. The electronic informationis transmitted in a standardized format over the computer network to theusers that have access to the information. In this way, the users canreadily adapt their medical diagnostic and treatment strategy inaccordance with the system's predictions which can be automaticallygenerated. Moreover, the system generates recommendations to usersregarding patient diagnosis and treatment.

In some embodiments, the described systems and methods are implementedas part of a digital and laboratory health care platform. The platformmay automatically generate a molecular report as part of a targetedmedical care precision medicine treatment. In some embodiments, thesystem in accordance with embodiments of the present disclosure operateson one or more micro-services, which can be micro-services of an ordermanagement system. In some embodiments, the system is implemented inconjunction with one or more micro-services of a cell-type profilingservice.

The feature modules 110 may store a collection of features, or statuscharacteristics, generated for some or all patients whose information ispresent in the system 100. These features may be used to generate andmodel predictions using the system 100. While feature scope across allpatients is informationally dense, a patient's feature set may besparsely populated across the entirety of the collective feature scopeof all features across all patients. For example, the feature scopeacross all patients may expand into the tens of thousands of features,while a patient's unique feature set may include a subset of hundreds orthousands of the collective feature scope based upon the recordsavailable for that patient.

A plurality of features present in the feature modules 110 may include adiverse set of fields available within patient health records 114.Clinical information may be based upon fields which have been enteredinto an electronic medical record (EMR) or an electronic health record(EHR) 116, which can be done automatically or manually, e.g., by aphysician, nurse, or other medical professional or representative. Otherclinical information may be curated information (115) obtained fromother sources, such as, for example, genetic sequencing reports (e.g.,from molecular fields). Sequencing may include next-generationsequencing (NGS) and may be long-read, short-read, or other forms ofsequencing a patient's somatic and/or normal genome. A comprehensivecollection of features (status characteristics) in additional featuremodules may combine a variety of features together across varying fieldsof medicine which may include diagnoses, responses to treatmentregimens, genetic profiles, clinical and phenotypic characteristics,and/or other medical, geographic, demographic, clinical, molecular, orgenetic features. For example, as shown in FIG. 1, a subset of featuresmay comprise molecular data features, such as features derived from anRNA feature module 111 or a DNA feature module 112 sequencing.

As further shown in FIG. 1, another subset of features, imaging featuresfrom imaging feature module 117, may comprise features identifiedthrough review of a specimen by pathologist, such as, e.g., a review ofstained H&E or IHC slides. As another example, a subset of features maycomprise derivative features obtained from the analysis of theindividual and combined results of such feature sets. Features derivedfrom DNA and RNA sequencing may include genetic variants from variantscience module 118, which can be identified in a sequenced sample.Further analysis of the genetic variants present in variant sciencemodule 118 may include steps such as identifying single or multiplenucleotide polymorphisms, identifying whether a variation is aninsertion or deletion event, identifying loss or gain of function,identifying fusions, calculating copy number variation, calculatingmicrosatellite instability, calculating tumor mutational burden, orother structural variations within the DNA and RNA. Analysis of slidesfor H&E staining or IHC staining may reveal features such as tumorinfiltration, programmed death-ligand 1 (PD-L1) status, human leukocyteantigen (HLA) status, or other immunology-related features.

Features derived from structured, curated, and/or electronic medical orhealth records 114 may include clinical features such as diagnosis,symptoms, therapies, outcomes, patient demographics such as patientname, date of birth, gender, ethnicity, date of death, address, smokingstatus, diagnosis dates for cancer, illness, disease, diabetes,depression, other physical or mental maladies, personal medical history,family medical history, clinical diagnoses such as date of initialdiagnosis, date of metastatic diagnosis, cancer staging, tumorcharacterization, tissue of origin, treatments and outcomes such as lineof therapy, therapy groups, clinical trials, medications prescribed ortaken, surgeries, radiotherapy, imaging, adverse effects, associatedoutcomes, genetic testing and laboratory information such as performancescores, lab tests, pathology results, prognostic indicators, date ofgenetic testing, testing provider used, testing method used, such asgenetic sequencing method or gene panel, gene results, such as includedgenes, variants, expression levels/statuses, or corresponding datesassociated with any of the above.

As shown in FIG. 1, the features 113 may be derived from informationfrom additional medical- or research-based Omics fields includingproteome, transcriptome, epigenome, metabolome, microbiome, and othermulti-omic fields. Features derived from an organoid modeling lab mayinclude the DNA and RNA sequencing information germane to each organoidand results from treatments applied to those organoids. Features 117derived from imaging data may further include reports associated with astained slide, size of tumor, tumor size differentials over timeincluding treatments during the period of change, as well as machinelearning approaches for classifying PDL1 status, HLA status, or othercharacteristics from imaging data. Other features may include additionalderivative features sets 119 derived using other machine learningapproaches based at least in part on combinations of any new featuresand/or those listed above. For example, imaging results may need to becombined with MSI calculations derived from RNA expressions to determineadditional further imaging features. As another example, amachine-learning model may generate a likelihood that a patient's cancerwill metastasize to a particular organ or a patient's future probabilityof metastasis to yet another organ in the body. Additional derivativefeature sets are discussed in more detail below with respect to FIG. 2.Other features that may be extracted from medical information may alsobe used. There are many thousands of features, and the above-describedtypes of features are merely representative and should not be construedas a complete listing of features.

In addition to the above features and enumerated modules, the featuremodules 110 may further include one or more of the modules that aredescribed below and that can be included within respective modules ofthe Feature modules 110, as a sub-module or as a standalone module.

Continuing with FIG. 1, a germline/somatic DNA feature module 112 maycomprise a feature collection associated with the DNA-derivedinformation of a patient and/or a patient's tumor. These features mayinclude raw sequencing results, such as those stored in FASTQ, BAM, VCF,or other sequencing file types known in the art; genes; mutations;variant calls; and variant characterizations. Genomic information from apatient's normal sample may be stored as germline and genomicinformation from a patient's tumor sample may be stored as somatic.

An RNA feature module 111 may comprise a feature collection associatedwith the RNA-derived information of a patient, such as transcriptomeinformation. These features may include, for example, raw sequencingresults, transcriptome expressions, genes, mutations, variant calls, andvariant characterizations. Features may also include normalizedsequencing results, such as those normalized by TMP.

The feature modules 110 can comprise various other modules. For example,a metadata module (not shown) may comprise a feature collectionassociated with the human genome, protein structures and their effects,such as changes in energy stability based on a protein structure.

A clinical module (not shown) may comprise a feature collectionassociated with information derived from clinical records of a patient,which can include records from family members of the patient. These maybe abstracted from unstructured clinical documents, EMR, EHR, or othersources of patient history. Information may include patient symptoms,diagnosis, treatments, medications, therapies, hospice, responses totreatments, laboratory testing results, medical history, geographiclocations of each, demographics, or other features of the patient whichmay be found in the patient's medical record. Information abouttreatments, medications, therapies, and the like may be ingested as arecommendation or prescription and/or as a confirmation that suchtreatments, medications, therapies, and the like were administered ortaken.

An imaging module, such as, e.g., the imaging module 117, may comprise afeature collection associated with information derived from imagingrecords of a patient. Imaging records may include H&E slides, IHCslides, radiology images, and other medical imaging information, as wellas related information from pathology and radiology reports, which maybe ordered by a physician during the course of diagnosis and treatmentof various illnesses and diseases. These features may include TMB,ploidy, purity, nuclear-cytoplasmic ratio, large nuclei, cell statealterations, biological pathway activations, hormone receptoralterations, immune cell infiltration, immune biomarkers of MMR, MSI,PDL1, CD3, FOXP3, HRD, PTEN, PIK3CA; collagen or stroma composition,appearance, density, or characteristics; tumor budding, size,aggressiveness, metastasis, immune state, chromatin morphology; andother characteristics of cells, tissues, or tumors for prognosticpredictions.

An epigenome module, such as, e.g., an epigenome module from Omicsmodule 113, may comprise a feature collection associated withinformation derived from DNA modifications which are not changes to theDNA sequence and regulate the gene expression. These modifications canbe a result of environmental factors based on what the patient maybreathe, eat, or drink. These features may include DNA methylation,histone modification, or other factors which deactivate a gene or causealterations to gene function without altering the sequence ofnucleotides in the gene.

A microbiome module, such as, e.g., a microbiome module from Omicsmodule 113, may comprise a feature collection associated withinformation derived from the viruses and bacteria of a patient. Thesefeatures may include viral infections which may affect treatment anddiagnosis of certain illnesses as well as the bacteria present in thepatient's gastrointestinal tract which may affect the efficacy ofmedicines ingested by the patient.

A proteome module, such as, e.g., a proteome module from Omics module113, may comprise a feature collection associated with informationderived from the proteins produced in the patient. These features mayinclude protein composition, structure, and activity; when and whereproteins are expressed; rates of protein production, degradation, andsteady-state abundance; how proteins are modified, for example,post-translational modifications such as phosphorylation; the movementof proteins between subcellular compartments; the involvement ofproteins in metabolic pathways; how proteins interact with one another;or modifications to the protein after translation from the RNA such asphosphorylation, ubiquitination, methylation, acetylation,glycosylation, oxidation, or nitrosylation.

Additional Omics module(s) (not shown) may also be included in Omicsmodule 113, such as a feature collection (which is a collection ofstatus characteristics) associated with all the different field ofomics, including: cognitive genomics, a collection of featurescomprising the study of the changes in cognitive processes associatedwith genetic profiles; comparative genomics, a collection of featurescomprising the study of the relationship of genome structure andfunction across different biological species or strains; functionalgenomics, a collection of features comprising the study of gene andprotein functions and interactions including transcriptomics;interactomics, a collection of features comprising the study relating tolarge-scale analyses of gene-gene, protein-protein, or protein-ligandinteractions; metagenomics, a collection of features comprising thestudy of metagenomes such as genetic material recovered directly fromenvironmental samples; neurogenomics, a collection of featurescomprising the study of genetic influences on the development andfunction of the nervous system; pangenomics, a collection of featurescomprising the study of the entire collection of gene families foundwithin a given species; personal genomics, a collection of featurescomprising the study of genomics concerned with the sequencing andanalysis of the genome of an individual such that once the genotypes areknown, the individual's genotype can be compared with the publishedliterature to determine likelihood of trait expression and disease riskto enhance personalized medicine suggestions; epigenomics, a collectionof features comprising the study of supporting the structure of genome,including protein and RNA binders, alternative DNA structures, andchemical modifications on DNA; nucleomics, a collection of featurescomprising the study of the complete set of genomic components whichform the cell nucleus as a complex, dynamic biological system;lipidomics, a collection of features comprising the study of cellularlipids, including the modifications made to any particular set of lipidsproduced by a patient; proteomics, a collection of features comprisingthe study of proteins, including the modifications made to anyparticular set of proteins produced by a patient; immunoproteomics, acollection of features comprising the study of large sets of proteinsinvolved in the immune response; nutriproteomics, a collection offeatures comprising the study of identifying molecular targets ofnutritive and non-nutritive components of the diet including the use ofproteomics mass spectrometry data for protein expression studies;proteogenomics, a collection of features comprising the study ofbiological research at the intersection of proteomics and genomicsincluding data which identifies gene annotations; structural genomics, acollection of features comprising the study of 3-dimensional structureof every protein encoded by a given genome using a combination ofmodeling approaches; glycomics, a collection of features comprising thestudy of sugars and carbohydrates and their effects in the patient;foodomics, a collection of features comprising the study of theintersection between the food and nutrition domains through theapplication and integration of technologies to improve consumer'swell-being, health, and knowledge; transcriptomics, a collection offeatures comprising the study of RNA molecules, including mRNA, rRNA,tRNA, and other non-coding RNA, produced in cells; metabolomics, acollection of features comprising the study of chemical processesinvolving metabolites, or unique chemical fingerprints that specificcellular processes leave behind, and their small-molecule metaboliteprofiles; metabonomics, a collection of features comprising the study ofthe quantitative measurement of the dynamic multiparametric metabolicresponse of cells to pathophysiological stimuli or genetic modification;nutrigenetics, a collection of features comprising the study of geneticvariations on the interaction between diet and health with implicationsto susceptible subgroups; cognitive genomics, a collection of featurescomprising the study of the changes in cognitive processes associatedwith genetic profiles; pharmacogenomics, a collection of featurescomprising the study of the effect of the sum of variations within thehuman genome on drugs; pharmacomicrobiomics, a collection of featurescomprising the study of the effect of variations within the humanmicrobiome on drugs; toxicogenomics, a collection of features comprisingthe study of gene and protein activity within particular cell or tissueof an organism in response to toxic substances; mitointeractome, acollection of features comprising the study of the process by which themitochondria proteins interact; psychogenomics, a collection of featurescomprising the study of the process of applying the powerful tools ofgenomics and proteomics to achieve a better understanding of thebiological substrates of normal behavior and of diseases of the brainthat manifest themselves as behavioral abnormalities, including applyingpsychogenomics to the study of drug addiction to develop more effectivetreatments for these disorders as well as objective diagnostic tools,preventive measures, and cures; stem cell genomics, a collection offeatures comprising the study of stem cell biology to establish stemcells as a model system for understanding human biology and diseasestates; connectomics, a collection of features comprising the study ofthe neural connections in the brain; microbiomics, a collection offeatures comprising the study of the genomes of the communities ofmicroorganisms that live in the digestive tract; cellomics, a collectionof features comprising the study of the quantitative cell analysis andstudy using bioimaging methods and bioinformatics; tomomics, acollection of features comprising the study of tomography and omicsmethods to understand tissue or cell biochemistry at high spatialresolution from imaging mass spectrometry data; ethomics, a collectionof features comprising the study of high-throughput machine measurementof patient behavior; and videomics, a collection of features comprisingthe study of a video analysis paradigm inspired by genomics principles,where a continuous digital image sequence, or a video, can beinterpreted as the capture of a single image evolving through time ofmutations revealing patient insights.

In some embodiments, a robust collection of features may include all ofthe features disclosed above. However, predictions based on theavailable features may include models which are optimized and trainedfrom a selection of fewer features than in an exhaustive feature set.Such a constrained feature set may include, in some embodiments, fromtens to hundreds of features. For example, a prediction may includepredicting the likelihood a patient's tumor may metastasize to thebrain. A model's constrained feature set may include the genomic resultsof a sequencing of the patient's tumor, derivative features based uponthe genomic results, the patient's tumor origin, the patient's age atdiagnosis, the patient's gender and race, and symptoms that the patientbrought to their physicians attention during a routine checkup. Examplesof optimized feature sets are further discussed below, in connectionwith FIGS. 3-5.

The feature store 120 may enhance a patient's feature set through theapplication of machine learning and/or an artificial intelligence engineand analytics by selecting from any features, alterations, or calculatedoutput derived from the patient's features or alterations to thosefeatures. One method for enhancing a patient's feature set may includedimensionality reduction, such as collapsing a feature set from tens ofthousands of features to a handful of features. Performingdimensionality reduction without losing information may be approached inan unsupervised manner or a supervised manner. Unsupervised methods mayinclude RNA Variational Auto-encoders, Singular Value Decomposition(SVD), PCA, KernelPCA, SparsePCA, DictionaryLearning, Isomap,Nonnegative Matrix Factorization (NMF), Uniform Manifold Approximationand Projection (UMAP), Feature agglomeration, Patient correlationclustering, KMeans, Gaussian Mixture, or Spherical KMeans. Performingdimensionality reduction in a supervised manner may include LinearDiscriminant Analysis, Neighborhood Component Analysis, MLP transferlearning, or tree based supervised embedding.

In one embodiment, a grid search may be performed across a variety ofencoding, such as the supervised and unsupervised approaches above,where each encoding is evaluated across a variety of hypertuningparameters to identify the encoding and hyperparameter set whichgenerates the highest dimensionality reduction while retaining orimproving accuracy.

In one embodiment, a grid search may identify a dimensionality reductionimplemented with tree-based supervised embedding on RNA TPM feature setsfor all patients. RNA TPM feature sets may be fit to a forest ofdecision trees, Such as a forest of decision trees generated fromhyperparameters of minimum samples per leaf using a minimum number of 2,4, 8, 16, 24, 100, or other selected number, a maximum feature set usinga percentage of the features which should be used in each tree, thenumber of trees to be used in the forest, and the number of clusterswhich may be identified from the reduced dimensionality dataset. Eachtree in the forest may randomly select up to the threshold percentage offeatures and with each selected feature identify the largest splitbetween patients who have metastasis and do not have metastasis. Whenthe feature set includes RNA TPM features, a random selection of genesmay include identifying which genes are the most divisive of the randomset of selected features, starting the branching from the most divisivegene and successively iterating down the gene list until either theminimum samples per leaf are not met or the maximum features are met.The leaf nodes for each tree include patients who meet the criteria ateach branch and are correlated based upon their likelihood tometastasize. Patient membership of each leaf may be evaluated usingone-hot KMeans cluster membership counts or a distance of each patientto each of the KMeans centroids/clusters.

In an example, the leaves of each tree are compared to identify whichleaves include the same branches or equivalent branches, such asbranches that result in the same patients because the genes, whiledifferent, are equivalent to each other. Equivalency may be determinedwhen information related to the expression level of a gene may becorrelated with, or predicted from, the expression level data associatedwith one or more other genes. When a gene may be correlated with, orpredicted from, one or more other genes, the one or more other genes aredefined as proxy genes. The terms proxy genes and equivalent genes maybe used interchangeably herein. Identifying the number of same branches,or equivalent branches, for each leaf allows generation of membershipfor each leaf as it occurs within the individual trees of the forest.Similarly, when KMeans clusters are generated from the collection ofleaves, a distance for each patient may be calculated for each patient.An array may be generated having the normalized inverse of each distancefor each patient to each KMeans centroid. The array, at this point, maybe stored as a reduced dimensionality feature set of RNA TPM featuresfor the set of patients, and the features of reduced dimensionality maybe used in any of the predictive methods described herein. In otherwords, the methods for identifying a prediction of a target/objectivepair may be performed having the array of distances for each patient asan input into the artificial intelligence engine described below;including, for example, performing logistic regression to generate apredictive model for a target/objective pair.

The feature store 120 may generate new features from the originalfeatures found in feature module 110 or may identify and store insightsor analysis derived using the features. The selections of features maybe based upon an alteration or calculation to be generated, and mayinclude the calculation of single or multiple nucleotide polymorphisms,insertion or deletions of the genome, a tumor mutational burden, amicrosatellite instability, a copy number variation, a fusion, or othersuch calculations. In an example, an output of an alteration modulewhich may inform future alterations or calculations may include afinding that patients having hypertrophic cardiomyopathy (HCM) expressvariants in MYH7 more commonly than patients without HCM. An exemplaryapproach may include the enrichment of variants and their respectiveclassifications to identify a region in MYH7 that is associated withHCM. Any novel variants detected from patient's sequencing localized tothis region would increase the patient's risk for HCM. Therefore,features which may be utilized in such an alteration detection includethe structure of MYH7, the normal genome for MYH7, and classification ofvariants therein as impacting a patient's chances of having HCM. A modelwhich focuses on enrichment may isolate such variants. Other variantsmay be isolated with respect to other illness, diseases, or diagnosisthrough an enrichment alteration module. The feature store selection,alteration, and calculations is discussed below in more detail withrespect to FIG. 2.

The feature generation 130 may process features from the feature store120 by selecting or receiving features from the feature store 120. Thefeatures may be selected based on a patient by patient basis, atarget/objective by patient basis, or a target/objective by all patientbasis, or a target/objective by cohort basis. In the patient by patientbasis, features which occur a specified patient's timeline of medicalhistory may be processed. In the target/objective by patient basis,features which occur in a specified patient's timeline which inform anidentified target/objective prediction may be processed. In someexamples, a model may be selected which optimizes the prediction basedupon the features available to the prediction engine at the time ofprocessing/generating a prediction for the patient or a prediction forall of the patients.

Targets/objectives may include a combination of an objective and ahorizon, or time period, such as Progression within 6, 12, 24, 60months, Death within 6, 12, 24, 60 months; Recurrence within 6,12, 24,60 months; First Administration of Medication within 7, 14, 21, or 28days; First Occurrence of Procedure within 7, 14, 21, or 28 days; FirstOccurrence of Adverse Reaction within 6, 12, or 24 months of InitialAdministration; Metastasis within 3 months; Metastasis to Organ within3, 6, 9, 12, or 24 months; Metastasis from Primary Organ Site toSecondary Organ Site (localized metastasis to an organ) within 3, 6, 9,12, or 24 months. The above listing of targets/objectives is notexhaustive, other objectives and horizons may be used based upon thepredictions requested from the system. In one example, the predictionmay be represented as P(Y(t) X), where P is the probability ofdeveloping a metastasis in organ Y at time t given the a patient'scurrent medical state and history X. Where the P includes atarget/objective, the X includes the patient features in the system. Inthe target/objective by all—patient basis, features which occur in eachpatient's timeline which inform an identified target/objectiveprediction may be processed for each patient until all patients havebeen processed. In the target/objective by cohort basis, features whichoccur in each patient's timeline which inform an identified targetprediction may be processed for each patient until all patients of acohort have been processed. A cohort may include a subset of patientshaving attributes in common with each other. For example, a cohort maybe a collection of patients which share a common institution (such as ahospital or clinic), a common diagnosis (such as cancer, depression, orother illness), a common treatment (such as a medication or therapy), orcommon molecular characteristics (such as a genetic variation oralteration). Cohorts may be derived from any feature or characteristicincluded in the feature modules 110 or feature store 120. Featuregeneration may provide a prior feature set and/or a forward feature setto a respective objective module corresponding to the target/objectiveand/or prediction to be generated. Prior and forward feature sets willbe disclosed in more detail with respect to FIGS. 3-5, below.

Objective Modules 140 may comprise a plurality of modules: ObservedSurvival 142, Progression Free Survival 144, Metastasis Site 146, andfurther additional models 148 which may include modules such asMedication or Treatment prediction, Adverse Response prediction, diseaseprogression, disease recurrence, or other predictive models. Each module142, 144, 146, and 148 may be associated with one or more targets 142 a,144 a, 146 a, and 148 a. For example, observed survival module 142 maybe associated with targets 142 a having the objective “Death” and timeperiods “6, 12, 24, and 60 months.” Progression free survival module 144may be associated with targets 144 a having the objective “Progression”and time periods “6, 12, 24, and 60 months.” Metastasis Site module 146may be associated with targets 146 a having the objective “Metastasis,Metastasis to Organ, Metastasis from Primary Organ Site to SecondaryOrgan Site” and time periods “3/6/9/12/24 months.” Additional models148, such as a Propensity Module may be associated with targets 148 ahaving an objective “Medications, Treatments, and Therapies” and timeperiods “7, 14, 21, and 28 days.” Additional models 148, such as aDisease Progression Module and Disease Recurrence Module may beassociated with targets 148 a having an objective “Progression,Recurrence” and time periods “6, 12, 24, and 60 months.” Each module142, 144, 146, and 148 may be further associated with models 142 b, 144b, 146 b, and 148 b. Models 142 b, 144 b, 146 b, and 148 b may begradient boosting models, random forest models, neural networks (NN),regression models, Naive Bayes models, or machine learning algorithms(MLA). A MLA or a NN may be trained from a training dataset such as aplurality of matrices having a feature vector for each patient or imagesand features. In an exemplary prediction profile, a training dataset mayinclude imaging, pathology, clinical, and/or molecular reports anddetails of a patient, such as those curated from an EHR or geneticsequencing reports. The training data may be based upon features such asthe objective specific sets disclosed with respect to FIGS. 3-5, below.

MLAs include supervised algorithms (such as algorithms where thefeatures/classifications in the dataset are annotated) using linearregression, logistic regression, decision trees, classification andregression trees, Naive Bayes, nearest neighbor clustering; unsupervisedalgorithms (such as algorithms where no features/classification in thedataset are annotated) using Apriori, means clustering, principalcomponent analysis, random forest, adaptive boosting; andsemi-supervised algorithms (such as algorithms where an incompletenumber of features/classifications in the dataset are annotated) usinggenerative approach (such as a mixture of Gaussian distributions,mixture of multinomial distributions, hidden Markov models), low densityseparation, graph-based approaches (such as mincut, harmonic function,manifold regularization), heuristic approaches, or support vectormachines. NNs include conditional random fields, convolutional neuralnetworks, attention based neural networks, deep learning, long shortterm memory networks, or other neural models where the training datasetincludes a plurality of tumor samples, RNA expression data for eachsample, and pathology reports covering imaging data for each sample.While MLA and neural networks identify distinct approaches to machinelearning, the terms may be used interchangeably herein. Thus, a mentionof MLA may include a corresponding NN or a mention of NN may include acorresponding MLA unless explicitly stated otherwise.

Training may include providing optimized datasets as a matrix of featurevectors for each patient, labeling these traits as they occur in patientrecords as supervisory signals, and training the MLA to predict anobjective/target pairing. Artificial NNs are powerful computing modelswhich have shown their strengths in solving hard problems in artificialintelligence. They have also been shown to be universal approximators(can represent a wide variety of functions when given appropriateparameters). Some MLA may identify features of importance and identify acoefficient, or weight, to them. The coefficient may be multiplied withthe occurrence frequency of the feature to generate a score, and oncethe scores of one or more features exceed a threshold, certainclassifications may be predicted by the MLA. A coefficient schema may becombined with a rule-based schema to generate more complicatedpredictions, such as predictions based upon multiple features. Forexample, ten key features may be identified across differentclassifications. A list of coefficients may exist for the key features,and a rule set may exist for the classification. A rule set may be basedupon the number of occurrences of the feature, the scaled weights of thefeatures, or other qualitative and quantitative assessments of featuresencoded in logic known to those of ordinary skill in the art.

In other MLAs, features may be organized in a binary tree structure. Forexample, key features which distinguish between the most classificationsmay exist as the root of the binary tree and each subsequent branch inthe tree until a classification may be awarded based upon reaching aterminal node of the tree. For example, a binary tree may have a rootnode which tests for a first feature. The occurrence or non-occurrenceof this feature must exist (the binary decision), and the logic maytraverse the branch which is true for the item being classified.Additional rules may be based upon thresholds, ranges, or otherqualitative and quantitative tests. While supervised methods are usefulwhen the training dataset has many known values or annotations, thenature of EMR/EHR documents is that there may not be many annotationsprovided. When exploring large amounts of unlabeled data, unsupervisedmethods are useful for binning/bucketing instances in the dataset. Asingle instance of the above models, or two or more such instances incombination, may constitute a model for the purposes of models 142 b,144 b, 146, and 148 b.

Models may also be duplicated for particular datasets which may beprovided independently for each objective module 142, 144, 146, and 148.For example, the metastasis site objective module 146 may receive a DNAfeature set, an RNA feature set, a combined RNA and DNA feature set, andobservational feature set, or a complete dataset comprising all featuresfor each patient. As another example, the metastasis site objectivemodule 146 may receive imaging features extracted from various digitalimages acquired from analysis of a patient's sample. A model 146 b maybe generated for each of the potential feature sets or targets 146 a.Each module 142, 144, 146, and 148 may be further associated withPredictions 142 c, 144 c, 146 c, and 148 c. A prediction may be a binaryrepresentation, such as a “Yes—Target predicted to occur” or “No—Targetnot predicted to occur.” Predictions may be a likelihood representationsuch as “target predicted to occur with 83% probability/likelihood.”Predictions may be performed on patient datasets having known outcomesto identify insights and trends which are unexpected. For example, acohort of patients may be generated for patients with a common cancerdiagnosis who have either remained progression free for five years afterdiagnosis, have progressed within five years after diagnosis, or whohave passed away within five years of diagnosis. A prediction model maybe associated with an objective for progression free survival (PFS) anda target of PFS within 2 years. The PFS model may identify every eventin each patient's history and generate a prediction of whether thepatient will be progression free within 2 years of that event. Thecohort of patients may generate, for each event in a patient's medicalfile, the probability that the patient will remain progression freewithin the next two years and compare that prediction with whether thepatient actually was progression free within two years of the event.

For example, a prediction that a patient may be progression free with a74% likelihood but in fact progresses within two years may inform theprediction model that intervening events before the progression areworth reviewing or prompt further review of the patient record that leadto the prediction to identify characteristics which may further inform aprediction. An actual occurrence of a target is weighted to 1 and thenon-occurrence of the event is weighted to 0, such that an event whichis likely to occur but does not may be represented by the difference(0-0.73), an event which is not likely to occur but does may berepresented by the difference (0.22-1), to provide a substantialdifference in values in comparison to events which are closely predicted(0-0.12 or 1-0.89) having a minimal difference. Predictions will bediscussed in further detail with respect to FIG. 6, below. Fordetermining a prediction, each module 142, 144, 146, and 148 may beassociated with a unique set of prior features, forward features, or acombination of prior features and forward features which may be receivedfrom feature generation 130. Selection of the unique set(s) of featureswill be disclosed in more detail with respect to FIGS. 3-5, below.

Prediction store 150 may receive predictions for targets/objectivesgenerated from objective modules 140 and store them for use in thesystem 100. Predictions may be stored in a structured format forretrieval by a user interface such as, for example, a webform-basedinteractive user interface which, in some embodiments, may includewebforms 160 a-n. Webforms may support GUIs that can be displayed by acomputer to a user of the computer system for performing a plurality ofanalytical functions, including initiating or viewing the instantpredictions from objective modules 140 or initiating or adjusting thecohort of patients from which the objective modules 140 may performanalytics from. Electronic reports 170 a-n may be generated and providedto the user via the graphical user interface (GUI) 165. It should beappreciated that the GUI 165 may be presented on a user device which isconnected to the content server/prediction engine 100 via a network.

The reports 170 can be provided to the user as part of a network-basedpatient management system that collects, converts and consolidatespatient information from various physicians and health-care providers(including labs) into a standardized format, stores it in network-basedstorage devices, and generates messages comprising electronic reportsonce the reports are generated in accordance with embodiments of thepresent disclosure. In this way, a user (e.g., a physician, oncologist,or any other health care provider, or a patient, receivescomputer-generated predictions related to a likelihood of a patient'stumor metastasizing, a predicted location of the metastasis, and/or anassociated timeline.

In some embodiments, the electronic report may include a recommendationto a physician to treat the patient using a treatment that correlateswith a magnitude of a determined degree of risk of the metastasis, arecommendation to a physician to de-escalate when the patient is lowrisk to reduce adverse events, save cost and improve health response, ora recommendation to a physician to elect a treatment which providesadjustments to the typical monitoring such as scanning, imaging, bloodtesting. Additionally or alternatively, the electronic report mayinclude a recommendation for accelerated screening of the patient, arecommendation for consideration of additional monitoring. In someembodiments, an electronic report indicating that a patient mayexperience metastasis to one or more predicted organs results inresearchers planning a clinical trial by predicting which groups ofpatients are most likely to respond to therapy that targets metastasesor recurrences in general or metastases to specific organ sites. In someembodiments, a clinical trial may be performed by selecting patients whoare predicted to be more likely or less likely to develop metastases orrecurrences in general or metastases to specific organ site, usingsystems and methods in accordance with the present disclosure.

FIG. 2 illustrates the generation of additional derivative feature sets119 of FIG. 1 and the feature store 120 using alteration modules. Afeature collection 205 may comprise the modules of feature modules 110,stored alterations 210 from the alteration module 250 and storedclassifications 230 from the structural variant classification 280. Analteration module 250 may be one or more microservices, servers,scripts, or other executable algorithms 252 a-n which generatealteration features associated with de-identified patient features fromthe feature collection. Exemplary alterations modules may include one ormore of the following alterations as a collection of alteration modules252 a-n. An SNP (single-nucleotide polymorphism) module may identify asubstitution of a single nucleotide that occurs at a specific positionin the genome, where each variation is present to some appreciabledegree within a population (e.g., >1%). For example, at a specific baseposition, or loci, in the human genome, the C nucleotide may appear inmost individuals, but in a minority of individuals, the position isoccupied by an A. This means that there is a SNP at this specificposition and the two possible nucleotide variations, C or A, are said tobe alleles for this position. SNPs underline differences insusceptibility to a wide range of diseases (e.g., sickle-cell anemia,β-thalassemia and cystic fibrosis result from SNPs).

The severity of illness and the way the body responds to treatments arealso manifestations of genetic variations. For example, a single-basemutation in the APOE (apolipoprotein E) gene is associated with a lowerrisk for Alzheimer's disease. A single-nucleotide variant (SNV) is avariation in a single nucleotide without any limitations of frequencyand may arise in somatic cells. A somatic single-nucleotide variation(e.g., caused by cancer) may also be called a single-nucleotidealteration. An MNP (Multiple-nucleotide polymorphisms) module mayidentify the substitution of consecutive nucleotides at a specificposition in the genome. An InDels module may identify an insertion ordeletion of bases in the genome of an organism classified among smallgenetic variations. While usually measuring from 1 to 10,000 base pairsin length, a microindel is defined as an indel that results in a netchange of 1 to 50 nucleotides. Indels can be contrasted with a SNP orpoint mutation. An indel inserts and deletes nucleotides from asequence, while a point mutation is a form of substitution that replacesone of the nucleotides without changing the overall number in the DNA.Indels, being either insertions, or deletions, can be used as geneticmarkers in natural populations, especially in phylogenetic studies.Indel frequency tends to be markedly lower than that of singlenucleotide polymorphisms (SNP), except near highly repetitive regions,including homopolymers and microsatellites. An MSI (microsatelliteinstability) module may identify genetic hypermutability (predispositionto mutation) that results from impaired DNA mismatch repair (MMR). Thepresence of MSI represents phenotypic evidence that MMR is notfunctioning normally. MMR corrects errors that spontaneously occurduring DNA replication, such as single base mismatches or shortinsertions and deletions. The proteins involved in MMR correctpolymerase errors by forming a complex that binds to the mismatchedsection of DNA, excises the error, and inserts the correct sequence inits place. Cells with abnormally functioning MMR are unable to correcterrors that occur during DNA replication and consequently accumulateerrors. This causes the creation of novel microsatellite fragments.Polymerase chain reaction-based assays can reveal these novelmicrosatellites and provide evidence for the presence of MSI.Microsatellites are repeated sequences of DNA. These sequences can bemade of repeating units of one to six base pairs in length. Although thelength of these microsatellites is highly variable from person to personand contributes to the individual DNA “fingerprint,” each individual hasmicrosatellites of a set length. The most common microsatellite inhumans is a dinucleotide repeat of the nucleotides C and A, which occurstens of thousands of times across the genome. Microsatellites are alsoknown as simple sequence repeats (SSRs). A TMB (tumor mutational burden)module may identify a measurement of mutations carried by tumor cellsand is a predictive biomarker being studied to evaluate its associationwith response to Immuno-Oncology (I-0) therapy. Tumor cells with highTMB may have more neoantigens, with an associated increase incancer-fighting T cells in the tumor microenvironment and periphery.These neoantigens can be recognized by T cells, inciting an anti-tumorresponse. TMB has emerged more recently as a quantitative marker thatcan help predict potential responses to immunotherapies across differentcancers, including melanoma, lung cancer and bladder cancer. TMB isdefined as the total number of mutations per coding area of a tumorgenome. Importantly, TMB is consistently reproducible. It provides aquantitative measure that can be used to better inform treatmentdecisions, such as selection of targeted or immunotherapies orenrollment in clinical trials. A CNV (copy number variation) module mayidentify deviations from the normal genome and any subsequentimplications from analyzing genes, variants, alleles, or sequences ofnucleotides. CNV are the phenomenon in which structural variations mayoccur in sections of nucleotides, or base pairs, that includerepetitions, deletions, or inversions.

A Fusions module may identify hybrid genes formed from two previouslyseparate genes. It can occur as a result of: translocation, interstitialdeletion, or chromosomal inversion. Gene fusion plays an important rolein tumorgenesis. Fusion genes can contribute to tumor formation becausefusion genes can produce much more active abnormal protein thannon-fusion genes. Often, fusion genes are oncogenes that cause cancer;these include BCR-ABL, TEL-AML1 (ALL with t(12;21)), AML1-ETO (M2 AMLwith t(8;21)), and TMPRSS2-ERG with an interstitial deletion onchromosome 21, often occurring in prostate cancer. In the case ofTMPRSS2-ERG, by disrupting androgen receptor (AR) signaling andinhibiting AR expression by oncogenic ETS transcription factor, thefusion product regulates the prostate cancer. Most fusion genes arefound from hematological cancers, sarcomas, and prostate cancer.BCAM-AKT2 is a fusion gene that is specific and unique to high-gradeserous ovarian cancer. Oncogenic fusion genes may lead to a gene productwith a new or different function from the two fusion partners.Alternatively, a proto-oncogene is fused to a strong promoter, andthereby the oncogenic function is set to function by an upregulationcaused by the strong promoter of the upstream fusion partner. The latteris common in lymphomas, where oncogenes are juxtaposed to the promotersof the immunoglobulin genes. Oncogenic fusion transcripts may also becaused by trans-splicing or read-through events. Since chromosomaltranslocations play such a significant role in neoplasia, a specializeddatabase of chromosomal aberrations and gene fusions in cancer has beencreated. This database is called Mitelman Database of ChromosomeAberrations and Gene Fusions in Cancer.

In some embodiments, an IHC (Immunohistochemistry) module may identifyantigens (proteins) in cells of a tissue section by exploiting theprinciple of antibodies binding specifically to antigens in biologicaltissues. IHC staining is widely used in the diagnosis of abnormal cellssuch as those found in cancerous tumors. Specific molecular markers arecharacteristic of particular cellular events such as proliferation orcell death (apoptosis). IHC is also widely used in basic research tounderstand the distribution and localization of biomarkers anddifferentially expressed proteins in different parts of a biologicaltissue. Visualizing an antibody-antigen interaction can be accomplishedin a number of ways. In the most common instance, an antibody isconjugated to an enzyme, such as peroxidase, that can catalyze acolor-producing reaction in immunoperoxidase staining. Alternatively,the antibody can also be tagged to a fluorophore, such as fluorescein orrhodamine in immunofluorescence. Approximations from RNA expressiondata, H&E slide imaging data, or other data may be generated. Forexample, in some embodiments, the predictions may include PD-L1prediction from H&E and/or RNA.

A Therapies module may identify differences in cancer cells (or othercells near them) that help them grow and thrive and drugs that “target”these differences. Treatment with these drugs is called targetedtherapy. For example, many targeted drugs go after the cancer cells'inner “programming” that makes them different from normal, healthycells, while leaving most healthy cells alone. Targeted drugs may blockor turn off chemical signals that tell the cancer cell to grow anddivide; change proteins within the cancer cells so the cells die; stopmaking new blood vessels to feed the cancer cells; trigger your immunesystem to kill the cancer cells; or carry toxins to the cancer cells tokill them, but not normal cells. Some targeted drugs are more “targeted”than others. Some might target only a single change in cancer cells,while others can affect several different changes. Others boost the wayyour body fights the cancer cells. This can affect where these drugswork and what side effects they cause.

In some embodiments, matching targeted therapies may include identifyingthe therapy targets in the patients and satisfying any other inclusionor exclusion criteria. A VUS (variant of unknown significance) modulemay identify variants which are called but cannot be classified aspathogenic or benign at the time of calling. VUS may be catalogued frompublications regarding a VUS to identify if they may be classified asbenign or pathogenic. A Trial module may identify and test hypothesesfor treating cancers having specific characteristics by matchingfeatures of a patient to clinical trials. These trials have inclusionand exclusion criteria that must be matched to enroll which may beingested and structured from publications, trial reports, or otherdocumentation.

An Amplifications module may identify genes which increase in countdisproportionately to other genes. Amplifications may cause a genehaving the increased count to go dormant, become overactive, or operatein another unexpected fashion. Amplifications may be detected at a genelevel, variant level, RNA transcript or expression level, or even aprotein level. Detections may be performed across all the differentdetection mechanisms or levels and validated against one another.

An Isoforms module may identify alternative splicing (AS), thebiological process in which more than one mRNA (isoforms) is generatedfrom the transcript of a same gene through different combinations ofexons and introns. It is estimated by large-scale genomics studies that30-60% of mammalian genes are alternatively spliced. The possiblepatterns of alternative splicing for a gene can be very complicated andthe complexity increases rapidly as the number of introns in a geneincreases. In silico alternative splicing prediction may find largeinsertions or deletions within a set of mRNA sharing a large portion ofaligned sequences by identifying genomic loci through searches of mRNAsequences against genomic sequences, extracting sequences for genomicloci and extending the sequences at both ends up to 20 kb, searching thegenomic sequences (repeat sequences have been masked), extractingsplicing pairs (two boundaries of alignment gap with GT-AG consensus orwith more than two expressed sequence tags aligned at both ends of thegap), assembling splicing pairs according to their coordinates,determining gene boundaries (splicing pair predictions are generated tothis point), generating predicted gene structures by aligning mRNAsequences to genomic templates, and comparing splicing pair predictionsand gene structure predictions to find alternative spliced isoforms.

A Pathways module may identify defects in DNA repair pathways whichenable cancer cells to accumulate genomic alterations that contribute totheir aggressive phenotype. Cancerous tumors rely on residual DNA repaircapacities to survive the damage induced by genotoxic stress which leadsto isolated DNA repair pathways being inactivated in cancer cells. DNArepair pathways are generally thought of as mutually exclusivemechanistic units handling different types of lesions in distinct cellcycle phases. Recent preclinical studies, however, provide strongevidence that multifunctional DNA repair hubs, which are involved inmultiple conventional DNA repair pathways, are frequently altered incancer. Identifying pathways which may be affected may lead to importantpatient treatment considerations. A Raw Counts module may identify acount of the variants that are detected from the sequencing data. ForDNA, this may be the number of reads from sequencing which correspond toa particular variant in a gene. For RNA, this may be the gene expressioncounts or the transcriptome counts from sequencing.

Structural variant classification 280 may evaluate features from featurecollection 205, alterations from alteration module 250, and otherclassifications from within itself from one or more classificationmodules 282 a-n. Structural variant classification 280 may provideclassifications to stored classifications 230 for storage. An exemplaryclassification module may include a classification of a CNV as“Reportable” may mean that the CNV has been identified in one or morereference databases as influencing the tumor cancer characterization,disease state, or pharmacogenomics, “Not Reportable” may mean that theCNV has not been identified as such, and “Conflicting Evidence” may meanthat the CNV has both evidence suggesting “Reportable” and “NotReportable.” Furthermore, a classification of therapeutic relevance issimilarly ascertained from any reference datasets mention of a therapywhich may be impacted by the detection (or non-detection) of the CNV.Other classifications may include applications of machine learningalgorithms, neural networks, regression techniques, graphing techniques,inductive reasoning approaches, or other artificial intelligenceevaluations within modules 282 a-n. A classifier for clinical trials mayinclude evaluation of variants identified from the alteration module 250which have been identified as significant or reportable, evaluation ofall clinical trials available to identify inclusion and exclusioncriteria, mapping the patient's variants and other information to theinclusion and exclusion criteria, and classifying clinical trials asapplicable to the patient or as not applicable to the patient. Similarclassifications may be performed for therapies, loss-of-function,gain-of-function, diagnosis, microsatellite instability, tumormutational burden, indels, SNP, MNP, fusions, and other alterationswhich may be classified based upon the results of the alteration modules252 a-n.

Each of the feature collection 205, alteration module 250, structuralvariant 280 and feature store 120 may be communicatively coupled to databus 290 to transfer data between each module for processing and/orstorage. In another embodiment, each of the feature collection 205,alteration module 250, structural variant 280 and feature store 120 maybe communicatively coupled to each other for independent communicationwithout sharing data bus 290.

FIGS. 3-5 illustrate the generation of feature sets from the featurestore on a target/objective basis. FIG. 3 illustrates a system 300 forretrieving a first subset 1-N of features from the feature store 120.Different targets and objective modules may perform optimally ondifferent feature sets. Feature selector and Prior feature set generatormay select features 1-N based on the provided target and objective toproduce an optimized, reduced feature set from which apatient-by-patient prior feature set may be generated. A prior featureset may be a collection of all features that occurred in a patienthistory before a specific date or may be an optimal collection of thebest representative set of features satisfying the input requirements ofa specific model, such as a model which has the best performance giventhe available features. For example, a patient with only DNA featuresmay have a likelihood of metastasis to an organ predicted from a modeltrained only on DNA features, whereas a patient with both DNA andclinical features may have a likelihood of metastasis to an organpredicted from a model trained on both DNA and clinical features. Inanother example, a patient having sparsely populated features ofnumerous models, such as RNA, DNA, and clinical features, may evaluateexpected performance from one or more combinations of the RNA, DNA, andclinical features alone and in combination to identify the best modeland the set of features generated may be reduced to those that fit theoptimal model. Other features, such as the specific date, may beselected from the current date at running of the model or any date inthe past. In an exemplary likelihood that a patient's cancer willmetastasize to a specifically identified organ in the patient's bodywithin a defined period of time prediction model, the specific date maybe an anchor point corresponding to the time of genetic sequencing at alaboratory, such as when a genetic sequencing laboratory providesresults of tumor sequencing. In some embodiments, the prior feature setmay be automatically analyzed and the most appropriate model may beselected based on the analysis.

Predictions may be effective tools for data science analytics to measurethe impact of treatments on the outcome of a patient's diagnosis,compare the outcomes of patients who took a medication against patientswho did not, or whether a patient will metastasize in a specified timeperiod. It may be advantageous to separate a patient information into acollection of distinct prior feature sets and forward feature sets suchthat at every time point in the patient's history, predictions may bemade and a more robust model generated that accurately predicts apatient's future satisfaction of a target/objective. A forward featureset may be advantageous when the predictive period for atarget/objective combination begins to exceed a period of time that newinformation may be entered into the system 300. For example, aprediction that a patient may take a medication in the next 16-25 dayshas a limited window for new information from the date of predictionsuch that the prediction is unlikely to change based on information thatbecomes available within the next 16-25 days. However, a prediction thata patient's cancer will remain progression-free for the next 24 monthsmay be greatly influenced by events that could happen in the next 24months. Therefore, an exemplary system 300 may generate a forwardfeature set which looks to events that may occur during the predictionperiod at feature generator 335. In one embodiment, feature pass-through340 may pass the prior feature set though the forward feature mapping330 to objective modules 140 without generating an accompanying forwardfeature set, for example, when the prediction is unlikely to be improvedby inclusion of a forward feature set.

As discussed above, the metastasis site objective module 146 may receivea DNA feature set, an RNA feature set, a combined RNA and DNA featureset, an observational feature set, partial subsets of features from thecomplete dataset, or a complete dataset comprising all features for eachpatient.

Various features may be generated and/or derived for a patient. Forexample, in some embodiments, the features can be related to RNA TPM(transcripts per million) count features. The feature space may compriseexpression levels of the RNA for some or all of the coding genes in thesample. The expression is assayed by counting the number of RNAmolecules (transcripts) that are present on a per gene basis. Tostandardize these counts across different experimental and technicalconditions, the counts per gene can be corrected by a normalizationfactor. This factor standardizes the expression data to represent thenumber of RNA molecules that would be associated with a single gene in apool of one million molecules, creating a TPM count.

In some embodiments, an input feature in a TPM space is a normalizedcount with a lower bound of 0, where the value represents the abundanceof the transcript. Transcripts over the whole exome (nearly 19K genes)can be considered. For example, in some embodiments, the genes compriseDPM1, SCYL3, C1orf112, FGR, CFH, FUCA2, GCLC, NFYA, STPG1, NIPAL3,LAS1L, ENPP4, SEMA3F, CFTR, ANKIB1, CYP51A1, and KRIT1.

In some embodiments, the features generated for a patient may includeRNA pathway features.

Previous experimental research has identified collections offunctionally related genes, which are stored and collected in the MSigDBMolecular Signatures Database. RNA pathway features can be generated byperforming single sample gene set enrichment analysis (ssGSEA) using thecollections of gene sets and individual sample gene expression rankings.ssGSEA acts by ranking the RNA expression within a sample and thenassigning a score to the gene set that is a function of that rank withinthe sample for the genes in the set. In practice, this functions to givehigh pathway scores to gene sets where all the genes in the set arehighly expressed in the sample, and vice versa for lowly expressedgenes. In practice, pathway scores serve to reduce some of the noise inthe RNA expression feature space.

In an example, an input feature in RNA Pathway space is a numericalvalue between −1 and 1 indicating the coincident expression, eitherup-regulated or down-regulated, of all of the genes in the pathwaygrouping. Non-limiting examples of the pathways include:HALLMARK_ADIPOGENESIS, HALLMARK_ALLOGRAFT_REJECTION,HALLMARK_ANDROGEN_RESPONSE, HALLMARK_ANGIOGENESIS,HALLMARK_APICAL_JUNCTION, HALLMARK_APICAL_SURFACE, HALLMARK_APOPTOSIS,HALLMARK_BILE_ACID_METABOLISM, HALLMARK_CHOLESTEROL_HOMEOSTASIS,HALLMARK_COAGULATION, HALLMARK_COMPLEMENT, HALLMARK_DNA_REPAIR,HALLMARK_E2F_TARGETS, HALLMARK_EPITHELIAL_MESENCHYMAL_TRANSITION,HALLMARK_ESTROGEN_RESPONSE_EARLY, and HALLMARK_ESTROGEN_RESPONSE_LATE.

In some embodiments, additionally or alternatively, the featurescomprise imaging features extracted from digital images. In someembodiments, imaging features can be divided into two categories:biologically meaningful features and geometrically meaningful features.Biologically meaningful features can include tumor percentage(percentage of the detected tissue area on the slide classified astumor, between 0 and 100), tumor cell percentage (percentage of thetotal cells that are tumor cells, as opposed to lymphocytes, between 0and 100), tumor infiltrating lymphocytes percentage (calculated as totalnumber of lymphocytes within the tumor region divided by total numbercells in the tumor region, between 0 and 100), tumor budding features(specifically for colorectal cancer, represented as integer counts fornumber of detected tumor buds, or normalized by density within a givenarea), and more. Geometrically meaningful features include aggregationmetrics (minimum, average, median, maximum) of tumor perimeter (tomeasure the perimeter of the tumor within a slide, given in pixels,where each pixel is 8 um by 8 um), average tumor cell circularity(calculated as cell area divided by the square of the perimeter,averaged over all cells, ranges between 0 and 1), average tumor celllength and aspect ratio (The eigenvalues are calculated for the cellshape, giving a relative, rotation independent, length and width of thecell; this is first done by identifying all pixels that are associatedwith the cell, and using the (x,y) pixel locations as points for acovariance matrix. The eigenvalues are then calculated, and the firstcomponent is taken as the length of the cell. The aspect ratio iscalculated as the second divided by the first eigenvalues, rangesbetween 0 and 1).

Referring back to FIG. 1, a model 146 b may be generated for each of thepotential feature sets or targets 146 a. FIG. 4 illustrates an exemplaryprior feature set which may be generated for a target/objectivecombination for predicting metastasis to brain within 24 months wherethe inputs narrowed to the prior features based on the target/objectiveof “metastasis to brain within 24 months—all features.” A sufficientlytrained model may identify a combination of features including cancersite, date since diagnosis, gender, symptoms, and sequencing informationas the most relevant features to predicting metastasis of a patient. Insome instances, a patient's tumor may be more likely to metastasize tothe brain when the originating tumor is an EGFR or HER2 positive lungcancer, a patient's tumor origin alone may influence metastasis when theorigin is a primary neoplasm such as melanoma, lung, breast, renal, andcolon cancer, the age of the patient may also play a role as childrenmay be more likely to metastasize than adults, a male patient with lungcancer may be more likely to metastasize, a female patient with breastcancer may also be more likely to metastasize, symptoms implicating thebrain from either neural discomfort such as headache, paresthesia ortingling in the patient's extremities, or a measurable increase inintercranial pressure may also increase the patient's likelihood formetastasis, and RNA/DNA sequencing results indicating a presence of aNOTCH2, FANCD2, EGFR, or TP53 variation or copy number change mayincrease a patient's likelihood for metastasis. Therefore, a predictivemodel may select a subset of features from the feature store 120including each of these features, and more, as identified by the optimalmodel given a patient's (or collection of patients') feature set(s).

FIG. 5 illustrates a prior feature selection set for a target/objectivepair metastasis to brain within 24 months using an observational model.In some embodiments, features of an observational model may be limitedto features which may be observed from patient results from tests,progress notes, but not medications, procedures, therapies, or otherproactive actions taken by a physician in treating the patient. Generalfeatures in the observational feature set may include a patient's age atevent for each event which may exist in the patient's record.Preprocessing steps may be performed on the ages available to reduce thedimensionality of the input features. For example, instead of having 100points for ages of patients, the patient's age may be fitted into agroup such as a range including 00 to 09, 10 to 19, 100 to 109, 110 to119, 20 to 29, 30 to 39, 40 to 49, 50 to 59, 60 to 69, 70 to 79, 80 to89, 90 to 99, or Unknown for each event in the patient's record. While abin of ten years is exemplified, other bin sizes may be used. Thereduction accomplished through binning features allows for a more robustanalysis of the bins rather than the granular age. The patient's genderor race may be normalized so that different sources having differentethnicity options are binned into similar ethnicities. For example, arace of Caucasian may be binned with white, a dataset includingJapanese, Korean, Phillipean distinctions may be binned into Asian, adataset with Hawaii, Guam, Tonga, Samoa, or Fiji may be binned intoPacific Islander, or a dataset with Cuban, Mexican, Puerto Rican, orSouth or Central American may be binned into Hispanic or Latino.Features which may be entered into the record by occurrence may betranslated and tracked by a number of days since the first or lastoccurrence. Days since the first or last occurrence features may includea tumor finding by histology for tumors including acinar_cell_carcinoma,adenocarcinoma,_no_subtype, carcinoma,_no_subtype,infiltrating_duct_carcinoma, lobular_carcinoma,malignant_neoplasm,_primary, mucinous_adenocarcinoma,neuroendocrine_carcinoma, non_small_cell_carcinoma, otherGroup,small_cell_carcinoma, small_cell_neuroendocrine_carcinoma,squamous_cell_carcinoma,_no_icd_o_subtype, ortransitional_cell_carcinoma. Other days since the first or lastoccurrence features may include tumor finding by histopathology grade orT-N-M stages including grade_1_(well_differentiated),grade_2_(moderately_differentiated), grade_3_(poorly_differentiated),grade_4_(undifferentiated), high_grade, m0, m1, mx, n0, n1, n2, n3, nx,pn0, pn1, pn2, pnx, stage_1, stage_2, stage_3, stage_4, pt1, pt2, pt3,pt4, t0, t1, t2, t3, t4, tx, or valg_stage-extensive.

Even other days since first or last occurrence features may includecancer type determinations or findings of breast, cervix_uteri, colon,head_and_neck, kidney, lung,lymphoid,_hemopoietic_and/or_related_tissue, otherGroup, ovary,pancreas, prostate, respiratory_tract, skin, skin_of_trunk,soft_tissues, stomach, tongue, unknown_site, or urinary_bladder. Stillfurther days since first or last occurrence features may include medicalevents, prior medications, or comorbidity or recurrence events includingemergency_room_admission, inpatient_stay,seen_in_hospital_outpatient_department,Abnormal_findings_on_diagnostic_imaging_of_breast,Administration_of_antineoplastic_agent, Anemia, Dehydration,Disorder_of_bone, Disorder_of_breast, Dyspnea, Essential_hypertension,Fatigue, Imaging_of_thorax_abnormal, Immunization_advised,Long_term_current_use_of_drug_therapy, Osteoporosis,Past_history_of_procedure, Screening_for_malignant_neoplasm_of_breast,chronic_obstructive_lung_disease, otherGroup, type_2_diabetes_mellitus,type_2_diabetes_mellitus_without_complication, emergency_room_admission,inpatient_stay, seen_in_hospital_outpatient_department, lung,otherGroup, or soft_tissues. DNA and RNA features which have beenidentified from a next generation sequencing (NGS) of a patient's tumoror normal specimen to identify germline or somatic variants includecategorizations of RNA expression analysis from an RNA auto encoder, DNArelated features (DNA variant calls) may include a calculation of themaximum effect a gene may have from sequencing results for the gene andsource set forth in Table 1, fluorescence_in_situ_hybridization_(fish),gene_mutation_analysis, gene_rearrangement_analysis, orimmunohistochemistry_(ihc) results. A patient's prior feature set may beselected from each of the above features identified within the patient'sstructured medical records available in the feature store 120. Priorfeature sets from the feature generator may be provided to thecorresponding model for the target/objective pair identified andpredictions generated for the patient.

FIG. 6 is a flow chart of a method 600 for generating prior feature setsand forward feature sets in accordance with some embodiments. At step610, the system may receive a set of data relating to one or morepatients, wherein the data can be obtained over time. The received setof data may include features from the feature generation 130 as arefined feature set described above with respect to FIGS. 4 and 5.Patient records are received which may span from a single entry todecades of medical records. While these records indicate the status ofthe patient over time, they may be received in a single transmission ora batch of transmissions. Each patient may have hundreds of records inthe system. An exemplary set of records for a patient may includephysician note entries from a routine doctor's visit where the doctorprescribed an antibiotic after determining the patient has a bacterialinfection, a scheduling request to see a specialist after the patientcomplained about headaches, scheduling request to take an MRI scan, anMM report summarizing the radiologists findings of an unknown mass inthe patient's lungs, a scheduling request to perform a biopsy of themass, a pathologist's report of the cells present in the biopsyspecimen, a prescription to begin a first line of therapy for lungcancer, an order for genetic sequencing of the biopsy specimen, and anysubsequent next-generation sequencing (NGS) report for the biopsyspecimen.

At step 620, the system may identify patient timepoints based on the setof data. Identified timepoints may include all timepoints from patientdiagnosis up to the last entry or patient's death. In sometarget/objective pairs, the only timepoint for identification is themost recent timepoint in which the patient received genetic sequencingresults, such as, e.g., results from a next-generation sequencer for thegenomic composition of the patient's tumor biopsy. An exemplarytimepoint selection for a metastasis to brain prediction may includeonly the date that the next-generation sequencing report for the biopsyspecimen was performed. In another embodiment, timepoint selection for apatient's likelihood to take undergo a progression event (an event fromwhich the cancer progresses such as metastasis, the tumor sizeincreases, or other events known to those of ordinary skill in the art)may include timepoints from records: the pathologist's report of thecells present in the biopsy specimen, the prescription to begin a firstline of therapy for lung cancer, the order for genetic sequencing of thebiopsy specimen, and the subsequent next-generation sequencing reportfor the biopsy specimen.

At step 630, the system may calculate outcome targets for a horizonwindow and outcome event. Outcome events may be the objectives, andhorizon windows may be the time periods such that an objective/targetpair is calculated. An exemplary target/objective pair may be metastasisto brain (the objective) within 24 months (the target). Thetarget/objective pair may also include the model from which the pairshould be calculated. An exemplary model may be an observation model.Other target/objective pairs, datasets, and models are introduced abovewith respect to objective modules 140. At step 640, the system mayidentify prior features and calculate the state of the prior features ateach timepoint. For example, for a target/objective pair “metastasis tobrain within 24 months—observational model,” as described above withrespect to FIG. 5, the set of prior features may be calculated once, atthe time of NGS. For a target objective pair “PFS within 2 years,” theset of prior features may be calculated for each timepoint correspondingto the following records: the pathologist's report of the cells presentin the biopsy specimen, the prescription to begin a first line oftherapy for lung cancer, the order for genetic sequencing of the biopsyspecimen, and the subsequent next-generation sequencing report for thebiopsy specimen.

At step 650 of FIG. 6, the system may identify forward features forevery horizon and outcome combination where the horizon is of asufficient duration that an event happening after the anchor point butbefore the termination of the timeline may have a noticeable effect onthe reliability of the prediction. A forward feature set may becalculated, at step 650, for horizons spanning months or years. In someembodiments, forward feature sets are calculated for horizons spanning acertain number of days. Forward features comprise the same feature setsas prior features but involve a conversion of the features from abackwards looking focus to a forwards looking focus. Exemplary forwardfeatures may include a computer-implemented determination of thefollowing: “Will patient take medication A after date of anchor pointand before date of endpoint,” “Will patient experience headaches afterdate of anchor point and before date of endpoint,” “Will patientprogress after date of anchor point and before date of endpoint,” or anyother forward looking version of features in the prior feature set.Forward features may be predicted using another target/objectiveprediction, ensemble model first, and the predictions themselves addedinto the feature set to influence the final prediction. For example, apatient who is observing increased intercranial pressure may bepredicted to experience headaches and a patient who experiences bothincreased intercranial pressure and headaches may be predicted to bemore likely to have metastasis to the brain. A model which finds that apatient with an increase in intercranial pressure is likely toexperience headaches within two weeks may provide additional featuresfrom which to inform the prediction of metastasis to the brain. Whilethe example is hypothetical, models may be trained to predict occurrenceof each feature.

FIG. 7 illustrates an exemplary timeline of events in a patient'smedical record which may provide prior features for a prior feature set.

A patient's medical record may have a unique series of events, orinteractions, as they face the challenges of rigoring through treatmentfor a disease. In patients who are diagnosed with cancer, some of theseevents may provide important features to prediction of a site of originof a metastatic tumor for the patient. For an exemplary patient, thefirst event informing their prior feature set may be a progress notefrom the date of diagnosis (Jan. 1, 2000) containing the patient'sinformation, cancer type, cancer stage, and other features. The secondevent informing their prior feature set may be a prescription formedications of a first line of therapy (Feb. 29, 2000) containing thepatient's medications, dosages, and expected administration frequency. Athird and fourth event may be a progress note from a physician whichnotes that an imaging scan of the tumor (Aug. 11, 2001) shows that ithas increased in size since the first line of therapy started and mayprompt the physician to prescribe medications for a second line oftherapy triggering another progress note (Sep. 12, 2001) containing thepatient's new medications, dosages, and expected administrationfrequency.

The final events, or interactions, in the patient's medical record priorto triggering a prediction of the patient's site-specific prediction ofmetastasis may include a physician's order for sequencing a biopsy ofthe tumor (Dec. 16, 2002) and a subsequent sequencing report (Jan. 24,2003) comprising the results of that sequencing. After a system, such asthe system of FIGS. 1 and 5 processing site-specific metastaticpredictions, including a metastasis to brain within 24 months, detectspresence of a stored sequencing report, a model pipeline may triggergeneration of the prediction. As another example, events, orinteractions, which trigger generation of a prediction may include aphysician's order for monitoring of the patient and a subsequent imagingreport comprising the results of that imaging, including MRI, X-Ray,radiology image, H&E slide, IHC Slide, or other imaging record.

In some embodiments, a model pipeline may include a plurality of models.When modeling with small sample sizes, random choice of specificpatients for hold-out set evaluation can have a large impact onresulting performance. With different train-test patient assignments, ahold-out set ROC AUC score can be, in some implementations, of from 0.3(considered to be worse than random) to 1.0 (considered to be a“perfect” model). In some embodiments, because of this large degree ofvariability, performance can be evaluated on a large number of differentpotential hold out sets, as opposed to relying on a single set ofpredefined train-test assignments.

In some embodiments, a modeling algorithm can include data preprocessing(log-transforming, one-hot encoding, imputing missing values, andin-line transformations such as z-scoring, dimensionality reductionmethods, etc.), robust feature selection (a bootstrapped approach usinglasso techniques, many different modifications of recursive featureelimination, Pearson correlation, correlated feature trimming, spectralbiclustering, or other methods, hyper-parameter tuning (model selectionfrom modifying the regularization strength in logistic regression, ornumber of estimators and maximum depth in a random forest, as examples),prediction generation (generating a probability between 0 and 1 for eachpatient at any given time horizon, from the tuned model), and featureimportance evaluation (where features are identified which are driving,or correlated with the prediction). It should be appreciated, however,that any variations of the modeling algorithm are possible.

In some embodiments of the present disclosure, the entire modelingalgorithm can be executed more than 100 times, each time with adifferent assignment of cross-validation folds and hold out set. Thisprocess results in over 100 out-of-fold cross validated scores on thetraining set and over 100 of hold-out (or test set) scores to allow formore robust evaluation of the model, given the chosen pipelineparameters, since it generates a distribution of performance metrics, asopposed to relying on single point estimates (which can have a largedegree of variance). This approach improves both model development andunderstanding of model generalizability. For the model development, thisallows us to more rigorously compare the potential benefit of change tothe pipeline (e.g., a new feature selection method, modeling framework,etc.), by comparing the two distributions of model performance scores,instead of comparing two held-out score point estimates. In terms ofmodel generalizability, the held-out score distribution gives a muchbetter understanding of how the model can be expected to perform oncompletely unseen data.

Furthermore, the large number of sets of predictions can also allowmaking some estimate of confidence about each patient's predictedprobability of metastasis, since the pipeline will generate the largenumber (e.g., at least 100, or at least 200, or at least 300, or atleast 400, or at least 500, or at least 1000) different predictions foreach patient, instead of only one single prediction. In addition, therepeated, multiple feature importance evaluations provide a more robustfeature importance analysis, because such approach allows selecting mostrobust features based not only on one specific training set, but in acertain percentage of the large number of different training sets. Athreshold can be used to determine which features are identified asrobust.

FIG. 8 illustrates an exemplary flowchart of a process 800 for applyinga model for predicting site-specific metastasis for a patient, inaccordance with some embodiments of the present disclosure. The process800 can be formed, for example, by the system 100 (FIG. 1) or by anothersuitable system.

At step 810, the system may receive target/objective pairs and priorfeature set for a cohort of patients. The system may also receive arequest to process one or more target/objective pairs from one or moreprior and forward feature sets. Each target/objective pair may bematched with a specific combination of prior and/or forward feature setsbased upon the requirements of a corresponding machine-learning model.At step 820, the system may identify metastatic sites to predict. In anembodiment, each of the target/objective pairs may reference a specificmetastasis site which may be passed through to model selection directly.In other embodiments, a target/objective pair may not specify ametastasis site—e.g., the target/objective pair may define a request topredict metastasis within 60 months. The system may then select a modeltrained for prediction of a certain metastasis site within the availablemodels, and it can pass the matched target/objective pair andcombination of prior and/or forward features to the model. At step 830,the system may receive prediction values for each patient of the cohortfor each metastatic site. The predictions may be stored in a predictionstore such as, e.g., the prediction store 150 or the predictions may bepassed to webforms for displaying prediction results for the patient ona graphical user interface of a computing device of a user. The user canbe, e.g., a patient's physician, oncologist, or another medicalprofessional. At step 840, the system may render, on the graphical userinterface of the computing device, in a graphical form, predictions ofmetastasis for a patient of the cohort. The predictions of metastasiscan be, e.g., in the format of a likelihood of metastasis to a certainpart/site of the body within a certain time period from the currenttime. The predictions can be displayed on the user interface inassociation with a computer-implemented representation of the human body(or its parts), or in other suitable format.

In some embodiments, the graph, images, and/or other information may begenerated in a corresponding webform for viewing the results ofsite-specific metastasis predictions. Metastasis predictions associatedwith the target/objective pair may be graphed on an image of a bodyand/or analytics may be viewed. Analytics may include the predictionpercentages, survival curves of the cohort, or features which weredriving factors in the prediction results generated. One example of awebform for displaying the graph is shown FIG. 9, discussed below.

Applications of predictions may include providing precision medicineresults for a patient. For example, a sample obtained from a patient maybe subjected to genetic sequencing during a course of treatment for acancer diagnosis. Predictions may be generated based upon the patient'sgenetic sequencing results, which provide insights on the patientsresponse to particular therapies. A physician may receive recommendedconsiderations as a component of a reporting of the genetic sequencingas a precision medicine result for the patient. Results may includetherapies which are expected to perform well for a patient havingcharacteristics similar to the reported patient, clinical trials whichmay accept the patient, or results of the sequencing which may influencethe physician's decisions. In one example, a patient may be prescribed atreatment which is considered aggressive for the treatment andprevention of metastasis. A prediction may be generated that thepatient, based upon their particular genetics and clinical history, areunlikely to metastasize to any localized region within the next 60months. A physician may then decide to suggest a less aggressivetreatment to the patient which may reduce the negative side effectsrelated to a harsher, more aggressive treatment and may be cheaper.

In another example, a patient may be prescribed an introductorytreatment which is not considered aggressive just to see how the patientresponds. A prediction may be generated that the patient, based upontheir particular genetics, clinical history, and most recent imagingreports are likely to metastasize from a primary cancer site to anotherlocalized region within the next 6 months. A physician may then decideto suggest a more aggressive treatment to reduce the chance that thepatient's tumor may metastasize to another localized region.Considerations made by the physician are not limited to treatments, as aphysician may utilize predictions to schedule the frequency ofmonitoring for the patient, such as follow-up visits, additionalscanning, screening, imaging, blood tests, or subsequent geneticsequencing. For example, a patient with a high prediction of metastasismay benefit from accelerated screening to detect changes as they occurrather than months after they occur and the patient is experiencingnoticeable side effects. In another example, a pharmaceutical companytesting a new drug may select potential test groups both off of theircurrent inclusion and exclusion criteria and the probability that thepatient will experience a predicted outcome.

In another example, a pharmaceutical company may retroactively analyzethe predicted outcome of patients in a clinical trial against how theyresponded to identify patient characteristics which may be included asinclusion or exclusion criteria in a future clinical trial. For example,patients which responded well to treatment and had a high prediction forsuccessful response to treatment may have features, or statuscharacteristics, in common which are absent from the patients which didnot respond well to treatment.

FIG. 9 illustrates an example of a webform for viewing site-specificpredictions of metastasis in a single patient. The webform can bedisplayed on a GUI of a user device (e.g., the GUI 165 of FIG. 1).

An exemplary webform may provide a patient portal to a user, such as,e.g., a physician, oncologist, or patient, that may request predictionsof metastasis based upon a target/objective scheme. For example, a usermay request a prediction of metastasis to the brain in the next 12months or a prediction of metastasis to any site in the next 60 months.The system, such as system 100 of FIG. 1, may either calculate aprediction on the fly or retrieve a precalculated prediction from theprediction store 150 and provide the webform with the predictioninformation for display to the user. In one embodiment a user mayrequest a prediction of metastasis to any site in 24 months. The webformmay receive the predictions and display them to the user through theuser interface of the webform.

The metastasis sites may be displayed in a number of different formats.A first format may include an image of a human body which regions havingmetastasis predictions highlighted therein. Highlighting for regionswith predictions may be color coded based upon the value of theprediction. For example, elements/organs/sites of the human body whichdo not have predictions may not be referenced in the image, such as thebreast or colon which are not referenced. A prediction falling below athreshold of 20% may receive a callout such as a line or other indicatorlinking the organ to the prediction threshold, such as the bones whichare referenced in the image with lines to the prediction value 16%. Aprediction falling between 20% and 50% may receive a callout linking theorgan to the prediction threshold and a color coded shading over theregion indicating the severity of the prediction, such as the liverwhich are referenced in the image with a line to the prediction value21% and a green shading over the region where a liver would be in ahuman. A prediction falling between 50% and 75% may receive a calloutlinking the organ to the prediction threshold and a color coded shadingover the region indicating the severity of the prediction, for example ayellow shading over the region where the metastasis site would be in ahuman. A prediction exceeding 75% may receive a callout linking theorgan to the prediction threshold and a color coded shading over theregion indicating the severity of the prediction, such as the brainwhich is referenced in the image with a line to the prediction value 77%and a red shading over the region where a brain would be in a human.

The above prediction ranges and combination of callout styles and colorshading are provided for illustrative purposes and are not intended tolimit the display to the user. Other combinations of prediction ranges,callout conventions, and/or coloring may be provided to the user withoutdeparting from the spirit of the disclosure. In addition to or as analternative to the first format, a second format may include a histogramor bar chart which provides a side by side comparison of the predictionsfor differing metastatic sites. For example, a lung cancer patient mayhave metastasis predictions for bone, brain, and liver sites. Ahistogram may display the predicted values of each side-by-side toprovide the user with a visual comparison of the likelihood ofmetastasis to each site. Other statistical, analytical, or graphicalrepresentations may be provided including charts, plots, and graphs suchas prediction distribution Kernel Density Estimate (KDE) plots, violinplots, per patient time series line plots of predicted likelihood ofmetastasis to patient organs over time, etc.

FIG. 10 illustrates elements of an exemplary webform 1000 for generatingsite-specific predictions of metastasis in a cohort of patients based ona user input received via the webform 1000.

An exemplary webform may provide a cohort portal to a user, such as aphysician, oncologist, or researcher may request predictions ofmetastasis based upon a target/objective scheme across an entire cohortof patients. For example, a user may request a prediction of metastasisto the brain in the next 12 months or a prediction of metastasis to anysite in the next 60 months. The system, such as system 100 of FIG. 1,may either calculate a prediction on the fly or retrieve a precalculatedprediction from the prediction store 150 and provide the webform withthe prediction information for display to the user. In one embodiment auser may request a prediction of metastasis to any site in 24 months.The webform may receive the predictions and display them to the userthrough the user interface of the webform. The receipt of the requestmay be facilitated through an aspect of the user interface containingone or more editable fields. For example, a first field may provide atext input or dropdown for selecting the origin site of cancer forpatents of the cohort. The origin site may be selected from anydiagnosable site of cancer, including: breast, lung, pancreas, prostate,colorectal, skin, brain, lymph nodes, and bone.

A second field may provide a text input or a drop down for selecting ametastasis site of cancer for patients in the cohort, including: breast,lung, pancreas, prostate, colorectal, skin, brain, lymph nodes, bone,and an “any” option to group all metastasis together. A third field mayprovide a text input or a drop down for selecting a horizon, or timeperiod, within which to predict the likelihood of metastasis forpatients in the cohort.

A fourth and fifth field may provide a text input or a drop down forselecting an anchor event and a corresponding anchor value. The anchorevent being the event that must be common across all patients in thecohort and from which the prediction's horizon will toll. Anchor eventsand corresponding values (presented below as Event: Values) may include:First Primary Cancer Diagnosis: Any Cancer Site (breast, lung, pancreas,prostate, colorectal, skin, brain, lymph nodes, bone, etc.); FirstStage: Any Cancer Stage (Stage 0, 1, 2, 3, 4); First medication: AnyMedication (doxorubicin, cyclophosphamide, anastrozole, tamoxifen,dexamethasone, pegfilgrastim, etc.); First Radiotherapy: AnyRadiotherapy Treatment (n-dimensional conformal radiation, cyberknife,external beam, image guided, intensity modulated, total body,radioactive isotope, etc.); First Procedure: Any Procedure (endoscopic,mastectomy, ablations, antrotomy, reconstructions, biopsies, excisions,resections, grafts, etc.); First Specimen Collection: Any Biopsy SiteFor Sequencing (breast, lung, pancreas, prostate, colorectal, skin,brain, lymph nodes, bone, etc.); First Alternative Grade: Any Grade(fuhrman stage 1-4, who stage i-iv, etc.); First Line of Therapy: AnyCombination of LoT Medications (abiraterone+apalutamide+leuprolide,abiraterone+ascorbic acid, fluorouracil+oxaliplatin,capecitabine+fulvestrant, etc.); and other combinations of events andvalues which may occur in a patient's medical record. A sixth andseventh field may provide a text input box and a button that whenactivated stores a copy of the above selected cohort restraints under aname entered into the textbox. Alternative means for storing the cohortmay be implemented in place of a text input field and button. Forexample, a single button may exist, which prompts a dialog box thatnavigates the file directory of the user's computer to select a locationand name for which to store the selections or no location may beavailable if the user is restricted to only storing the saved cohortselections on the server for online-access only.

Selecting a cancer origin site, a cancer metastasis site, an anchorevent, and/or a survival curve group may further filter the cohort toonly patients which have the respective prerequisite event or outcome intheir patient records, or those patients who receive the selectedprediction.

The metastasis sites may be displayed in a number of different formats.In some embodiments, for example, a format may include an image of ahuman body which regions having metastasis predictions highlightedtherein. Highlighting for regions with predictions may be color codedbased upon the value of the prediction. For example,elements/organs/sites of the human body which do not have predictionsmay not be referenced in the image, such as the breast or colon whichare not referenced. A prediction falling below a threshold of 20% mayreceive a callout such as a line or other indicator linking the organ tothe prediction threshold, such as the bones which are referenced in theimage with lines to the prediction value 16%.

A prediction falling between 20% and 50% may receive a callout linkingthe organ to the prediction threshold and a color coded shading over theregion indicating the severity of the prediction, such as the liverwhich are referenced in the image with a line to the prediction value21% and a green shading over the region where a liver would be in ahuman. A prediction falling between 50% and 75% may receive a calloutlinking the organ to the prediction threshold and a color coded shadingover the region indicating the severity of the prediction, for example ayellow shading over the region where the metastasis site would be in ahuman.

A prediction exceeding 75% may receive a callout linking the organ tothe prediction threshold and a color coded shading over the regionindicating the severity of the prediction, such as the brain which isreferenced in the image with a line to the prediction value 77% and ared shading over the region where a brain would be in a human. The aboveprediction ranges and combination of callout styles and color shadingare provided for illustrative purposes and are not intended to limit thedisplay of such to the user. Other combinations of prediction ranges,callout conventions, and/or coloring may be provided to the user withoutdeparting from the spirit of the disclosure. In addition to or as analternative to the first format, a second format may include a histogramor bar chart which provides a side by side comparison of the predictionsfor differing metastatic sites. For example, a cohort of lung cancerpatients may have metastasis predictions for bone, brain, liver, lymphnode, other and any sites. A histogram may display the predicted valuesof each side-by-side to provide the user with a visual comparison of thelikelihood of metastasis to each site. Additionally, a set of histogramsmay be viewed together, one for each of a set of horizons. For example,a first histogram may display the cohort average predictions for ahorizon of 6 months, a second histogram for a horizon of 12 months, athird histogram for a horizon of 24 months, a fourth histogram for ahorizon of 60 months, and so on. In addition to, or as an alternative tothe first or second format, prediction distributions graphs, survivalcurves, or Kaplan Meier plots may be considered. Other statistical,analytical, or graphical representations may be provided includingcharts, plots, and graphs.

Once a user has accessed the webform, requested predictions ofmetastasis based upon a target/objective scheme across an entire cohortof patients, and obtained the displayed predictions via the userinterface of the webform, the user may desire to understand whichfeatures shared by members of the cohort were most influential indriving the predictions and facilitate model interpretability. The mostinfluential features can be, for example, features related to or derivedfrom sequencing information, such as information on genes that are mostinformative of the generated predictions. An adaptive algorithm runsalongside the modeling to generate viable feature importance ranksexclusively on the selected sub-population of patients without needingto re-train the underlying models. An exemplary adaptive algorithm may:calculate population mean prediction across the patients in the cohort;encode categorical feature levels, including clustering/bucketingcontinuous features, as the difference/delta between the predicted valueand the population mean prediction; aggregate average probabilitydifference with the estimated percentage per categorical level andassign overall feature importance as the frequency-weighted sum ofabsolute value of all values; and assign an impact value representingeach feature's co-occurrence with an observed deviation from predictionmean to explore the variation in impact per change in feature value. Agraphical representation of the feature enrichment ranking results maybe presented according to an embodiment of FIGS. 11 and 12.

FIG. 11 illustrates elements of an exemplary webform for viewing featureimportance rankings of site-specific predictions of metastasis in acohort of patients.

A first field may provide a text input, radio button, toggle, or a dropdown for selecting a feature importance ranking visualization method forselecting between a heatmap, feature enrichment presentation and ascaled, ranking bar feature importance representation. One or moreadditional radio buttons, toggles, or other feature selectors may bepresented to the user to allow the selection of which features should beincluded in the feature importance model. Selectable features mayinclude any level of categorization of the features in the inputdataset, including patient demographics, germline results fromsequencing, cancer types and/or stages, procedures or radiotherapiesunderwent by the patient, genomic or sequencing results of the patient'stumor or normal specimens, or medications taken by the patient.Selection of a selectable feature will trigger the inclusion orexclusion of the associated features from the feature importancecalculations and the remaining features' weights will be recalculated tocompensate for the adjustment to features.

An exemplary feature enrichment graphical representation may provide aheatmap of the feature importance to each model prediction ofmetastasized or did not metastasize. The heatmap may be selected betweenone or more colors such that if a single color is used in the heatmapvisualization, the intensity of the color may vary to indicate astronger or weaker importance of the feature in determining the model'sprediction. The heatmap may be selected between two or more colors suchthat if multiple colors are used in the heatmap visualization, the colorselection may vary to indicate a stronger or weaker importance of thefeature in determining the model's prediction. The heatmap may beselected between two or more colors such that if multiple colors areused in the heatmap visualization, the color selection may vary toindicate a stronger or weaker importance of the feature in determiningthe model's prediction and the intensity of the color may furtherprovide ranking visualizations within each classification of the featureimportance. For example, a green color may be used for features whichare most important to the model for predicting metastasize, a red colormay be used for features which are most important to the model forpredicting did not metastasize, and a yellow color may be used tofeatures which were relevant to either metastasize or did notmetastasize but were not the most significant of drivers in theprediction.

Further, within each classification color of green, red, and yellow, theintensity of the color may rank the importance of the features in eachcategory such that light intensity corresponds with features of theleast importance and bright, bold colors corresponds with features ofthe most importance. In addition to the color and intensity selection, apercentage of the patients in the cohort which presented the feature andwere predicted to have metastasized or did not metastasized may beprovided in the color coding of the feature. For example, a first columnmay be provided for prediction-metastasized features and a second columnmay be provided for prediction-did_not_metastasize features. Each row ofthe two columns may correspond to a single feature. The featureshierarchically organized into the ranking of the features by importanceto the predictions. A first feature may represent the greatestdetermining factor in the prediction of metastasized and did notmetastasize and may be “cancer stage 3 or greater.” 40% of the patientswho were predicted to have metastasized had stage 3 cancer or greaterwhile only 4% of patients who were not predicted to have metastasizedhad stage 3 cancer or greater. Because 40% is substantially greater than4% the intensity of the coloring may be higher for the 40% heatmap andlower for the 4% heatmap. Another feature of the heatmap,“BRIP1-germline: moderate” may be one of the top 20 features relied onby the predictions with 58% of the patients who were predicted to havemetastasized presenting the feature and 73% of the patients who werepredicted to not metastasize presenting the feature. Because 58% isgreater than 40% the intensity of the color may be even greater than the40% heatmap and the intensity of the 73% even greater still.

FIG. 12 illustrates elements of an exemplary webform for viewing featureimportance rankings of site-specific predictions of metastasis in acohort of patients.

When the first field for selecting a feature importance rankingvisualization method has the scaled, ranking feature importance barrepresentation selected, an exemplary feature importance graphicalrepresentation may provide a ranked, bar chart of the feature importanceto each model prediction of metastasized or did not metastasize. The barchart may be selected between two colors, a first color forprediction-metastasized feature importance and a second color forprediction-did_not_metstasize feature importance. The length of the barmay correspond to the number of patients in the cohort which presentedthe feature and were predicted to have metastasized or did notmetastasized. For example, each feature may be hierarchically organizedby rows into the ranking of the features by importance to thepredictions. A first color may identify features which are mostimportant for predicting metastasized and a second color may identifyfeatures which are most important for predicting did not metastasize. Afirst row may identify the first feature and may represent the greatestdetermining factor in the prediction of metastasized and did notmetastasize and may be “cancer stage 3 or greater.” The feature may,based upon the results of the adaptive algorithm, have the bar with thegreatest length to visually represent the feature's importance and thefirst color to indicate that the feature weighs most towardmetastasized. A second row may identify the second feature and mayrepresent the greatest determining factor in the prediction ofmetastasized and did not metastasize and may be “took_medication:heparin.” The feature may, based upon the results of the adaptivealgorithm, have the bar with the second greatest length to visuallyrepresent the feature's importance and the second color to indicate thatthe feature weighs most toward did not metastasize. Features continuingdown the list may have increasingly shorter bars of either the first orsecond color to indicate their respective weights for or against thepredictions for metastasized.

FIG. 13 is an illustration 1300 of exemplary aggregate measures ofperformance across possible classification thresholds of input datasetsaccording to an objective of predicting metastasis in lung cancerpatients to any other cancer site within 24 months.

As discussed above with respect to FIG. 1, there are a number of modelswhich may be selected and for each model there are a number of tuningparameters which may be considered. For an objective of metastasisprediction the collection of sites to which the patient will metastasizewithin the specified time horizon (24 months) at each time point may beused as the target of interest. The metastasis sites which may beconsidered include breast, colon, lung, liver, bone, brain and lymphnode, with any other sites being grouped into a miscellaneous category.Other combinations of metastasis sites may be considered as well. Duringpreprocessing, it may be advantageous to impose an additionalrequirement that each target must have more than one unique value withinevery cross validation fold in order to ensure the sites at whichpredictions are generated are variable depending on the origin cancersite.

Given a curated dataset with the five most common cancers in a cohort ofall metastasized cancers being ovary, prostate, colon, breast, and lung,it may be advantageous to tune a multilabel random forest using 4batches of 5 jobs, optimizing the average area under curve (AUC) acrossall target labels. In general, the models seem to prefer a large numberof deep trees with heavy column sampling at each split, which could beused to improve future tuning jobs.

As an example, a random forest-based model may be instantiated using thefollowing parameters:

range of max_depth: (5, 23)

n_estimators: (100, 1000)

min_samples_leaf: (20, 200)

max_features:(0.5, 0.8).

The following performance scores can be derived from the model basedupon a set of patients for training according to cancer diagnosis.

An ovary objective scores by metastasis site may be:

Lymph node: 0.831445

Lung: 0.768152

An ovary predicted parameter set may be:

max_depth: 23

max_features: 0.70

min_samples_leaf: 58

n_estimators: 329

A prostate objective scores by metastasis site may be:

Lymph node: 0.784173

Other site: 0.784805

Bone: 0.878749

A prostate predicted parameter set may be:

max_depth: 15

max_features: 0.50

min_samples_leaf: 53

n_estimators: 748

A colon objective scores by metastasis site may be:

Lymph node: 0.836868

Liver: 0.877584

Other site: 0.840575

Lung: 0.885678

A colon predicted parameter set may be:

max_depth: 19

max_features: 0.57

min_samples_leaf: 55

n_estimators: 923

A breast objective scores by metastasis site may be:

Lymph node: 0.810405

Liver: 0.883235

Other site: 0.819709

Brain: 0.807003

Bone: 0.852316

Lung: 0.798472

A breast predicted parameter set may be:

max_depth: 23

max_features: 0.52

min_samples_leaf: 119

n_estimators: 821

A lung scores by metastasis site may be:

Lymph node: 0.725858

Liver: 0.840760

Other site: 0.771431

Brain: 0.791871

Bone: 0.724428

A lung predicted parameter set may be:

max_depth: 22

max_features: 0.51

min_samples_leaf: 111

n_estimators: 344

Given a known set of hyperparameters for each objective, such as thoselisted above, it may be advantageous to consider the impacts of aselected feature set for each objective. For example, a feature set forDNA related features (DNA variant calls) may include a calculation ofthe maximum effect a gene may have from sequencing results for the geneand source set forth in Table 1. A max effect calculation may includeidentifying an integer in a range from 0 to 7, wherein a 0 represents noeffect and a 7 represents the highest effect a gene may impact apatient's cancer diagnosis. While the values 0-7 are used forillustrative purposes, other values may be used according to a desiredresolution for measuring the effect. The values may be classified from avariant science pipeline based upon a characterization of the varianteffect as pathogenic, benign, or unknown. In one example, a varianthaving a pathogenic classification may be assigned a value of 7 where avariant having a benign classification may be assigned a value of 0.Values of differing degrees may be awarded when mitigating oraggravating factors are present. For example, a variant which hassubstantial documentation within the medical committee for causingcancer may be assigned a higher value than a variant which has nominaldocumentation within the medical community for causing cancer. In oneexample, genetic variants are assigned a max effect value and a modelmay be trained on a variant by variant basis. A variant by variant modelmay be trained on variant max effects and a supervisory signalidentifying patient metastasis. In another example, genetic variants areassigned a max effect value, but a model may be trained on a gene bygene basis. Converting variant max effect into gene max effect mayinclude a number of approaches such as taking the highest max effect orapplying customized weights to each max effect based upon the number ofreads associated with the variant from sequencing of the patient'stumor. In one example, where the highest max effect is assigned,variants for each gene are compared to identify the highest max effectrelating to the gene, and the highest max effect is assigned to thegene. Where the max effects are provided a customized weighting schema,each variant may be assigned a weight to scale the max effect and thosemax effects are combined into a gene max effect. For example, a genewith four identified variants may scale each max effect by 0.25 and sumthe combined, scaled max effects into a gene max effect, effectivelyaveraging the max effects. In another aspect, a gene with four variantshaving raw reads of 25, 100, 250, and 75 may scale each max effect by25/450, 100/450, 250/450, and 75/450 respectively. A gene with no calledvariants (variants identified in the patient's genome) for a particulargene is assigned a max effect of 0.

TABLE 1 ABCB1-somatic ACTA2-germline ACTC1-germlineALK-fluorescence_in_situ_hybridization_(fish)ALK-immunohistochemistry_(ihc) ALK-md_dictated ALK-somatic AMER1-somaticAPC-gene_mutation_analysis APC-germline APC-somatic APOB-germlineAPOB-somatic AR-somatic ARHGAP35-somatic ARID1A-somatic ARID1B-somaticARID2-somatic ASXL1-somatic ATM-gene_mutation_analysis ATM-germlineATM-somatic ATP7B-germline ATR-somatic ATRX-somatic AXIN2-germlineBACH1-germline BCL11B-somatic BCLAF1-somatic BCOR-somatic BCORL1-somaticBCR-somatic BMPR1A-germline BRAF-gene_mutation_analysis BRAF-md_dictatedBRAF-somatic BRCA1-germline BRCA1-somatic BRCA2-germline BRCA2-somaticBRD4-somatic BRIP1-germline CACNA1S-germline CARD11-somatic CASR-somaticCD274-immunohistochemistry_(ihc) CD274-md_dictated CDH1-germlineCDH1-somatic CDK12-germline CDKN2A-immunohistochemistry_(ihc)CDKN2A-germline CDKN2A-somatic CEBPA-germline CEBPA-somatic CFTR-somaticCHD2-somatic CHD4-somatic CHEK2-germline CIC-somatic COL3A1-germlineCREBBP-somatic CTNNB1-somatic CUX1-somatic DICER1-somatic DOT1L-somaticDPYD-somatic DSC2-germline DSG2-germline DSP-germline DYNC2H1-somaticEGFR-gene_mutation_analysis EGFR-immunohistochemistry_(ihc)EGFR-md_dictated EGFR-germline EGFR-somatic EP300-somatic EPCAM-germlineEPHA2-somatic EPHA7-somatic EPHB1-somaticERBB2-fluorescence_in_situ_hybridization_(fish)ERBB2-immunohistochemistry_(ihc) ERBB2-md_dictated ERBB2-somaticERBB3-somatic ERBB4-somatic ESR1-immunohistochemistry_(ihc) ESR1-somaticETV6-germline FANCA-germline FANCA-somatic FANCD2-germlineFANCI-germline FANCL-germline FANCM-somatic FAT1-somatic FBN1-germlineFBXW7-somatic FGFR3-somatic FH-germline FLCN-germline FLG-somaticFLT1-somatic FLT4-somatic GATA2-germline GATA3-somatic GATA4-somaticGATA6-somatic GLA-germline GNAS-somatic GRIN2A-somatic GRM3-somaticHDAC4-somatic HGF-somatic IDH1-somatic IKZF1-somatic IRS2-somaticJAK3-somatic KCNH2-germline KCNQ1-germline KDM5A-somatic KDM5C-somaticKDM6A-somatic KDR-somatic KEAP1-somatic KEL-somatic KIF1B-somaticKMT2A-fluorescence_in_situ_hybridization_(fish) KMT2A-somaticKMT2B-somatic KMT2C-somatic KMT2D-somatic KRAS-gene_mutation_analysisKRAS-md_dictated KRAS-somatic LDLR-germline LMNA-germline LRP1B-somaticMAP3K1-somatic MED12-somatic MEN1-germlineMET-fluorescence_in_situ_hybridization_(fish) MET-somaticMKI67-immunohistochemistry_(ihc) MKI67-somatic MLH1-germlineMSH2-germline MSH3-germline MSH6-germline MSH6-somatic MTOR-somaticMUTYH-germline MYBPC3-germline MYCN-somatic MYH11-germline MYH11-somaticMYH7-germline MYL2-germline MYL3-germline NBN-germline NCOR1-somaticNCOR2-somatic NF1-somatic NF2-germline NOTCH1-somatic NOTCH2-somaticNOTCH3-somatic NRG1-somatic NSD1-somatic NTRK1-somatic NTRK3-somaticNUP98-somatic OTC-germline PALB2-germline PALLD-somatic PBRM1-somaticPCSK9-germline PDGFRA-somatic PDGFRB-somaticPGR-immunohistochemistly_(ihc) PIK3C2B-somatic PIK3CA-somaticPIK3CG-somatic PIK3R1-somatic PIK3R2-somatic PKP2-germline PLCG2-somaticPML-somatic PMS2-germline POLD1-germline POLD1-somatic POLE-germlinePOLE-somatic PREX2-somatic PRKAG2-germline PTCH1-somaticPTEN-fluorescence_in_situ_hybridization_(fish)PTEN-gene_mutation_analysis PTEN-germline PTEN-somatic PTPN13-somaticPTPRD-somatic RAD51B-germline RAD51C-germline RAD51D-germlineRAD52-germline RAD54L-germline RANBP2-somatic RB1-germline RB1-somaticRBM10-somatic RECQL4-somaticRET-fluorescence_in_situ_hybridization_(fish) RET-germline RET-somaticRICTOR-somatic RNF43-somaticROS1-fluorescence_in_situ_hybridization_(fish) ROS-md_dictatedROS1-somatic RPTOR-somatic RUNX1-germline RUNX1T1-somatic RYR1-germlineRYR2-germline SCN5A-germline SDHAF2-germline SDHB-germline SDHC-germlineSDHD-germline SETBP1-somatic SETD2-somatic SH2B3-somatic SLIT2-somaticSLX4-somatic SMAD3-germline SMAD4-germline SMAD4-somatic SMARCA4-somaticSOX9-somatic SPEN-somatic STAG2-somatic STK11-gene_mutation_analysisSTK11-germline STK11-somatic TAF1-somatic TBX3-somatic TCF7L2-somaticTERT-somatic TET2-somatic TGFBR1-germline TGFBR2-germline TGFBR2-somaticTMEM43-germline TNNI3-germline TNNT2-germlineTP53-gene_mutation_analysis TP53-immunohistochemistry_(ihc)TP53-md_dictated TP53-germline TP53-somatic TPM1-germline TSC1-germlineTSC1-somatic TSC2-germline VHL-germline WT1-germline WT1-somaticXRCC3-germline ZFHX3-somatic

In some examples, for a metastatic location prediction model trained onDNA features only, a resulting receiver operating characteristic (ROC)area under curve (AUC) may be approximately 0.52.

A feature set for RNA related features may include features associatedwith raw read counts for every transcriptome of the human genome,features associated with normalized read counts for every transcriptomeof the human genome, or features associated with normalized, encodedread counts, such as encoded via an autoencoder or a dimensionalityreducer. Raw read counts may be accompanied by a normal value,identifying the expected number of read counts should the transcriptomebe normally expressed. Raw read counts exceeding the normal value may beconsidered over expressed, and raw read counts falling below the normalvalue may be considered under expressed. Normalized read counts may benormalized to ensure that while every transcriptome has its own normalvalue, the resulting normalized value falls within a desired range thataccounts for the differences between each unnormalized transcriptomsnormal. For example, RPKM (Reads Per Kilobase Million), FPKM (FragmentsPer Kilobase Million), or TPM (Transcripts Per Kilobase Million) may beused for normalization. RPKM may be calculated by scaling the total RNAreads of a specimen by 1,000,000 to create a scaling factor, scaling thetotal reads for any read counts for each read by the scaling factor tocreate an RPM, and dividing the RPM by the length of the gene to createan RPKM. FPKM may be generated by performing the same steps, but whenperforming pair-end sequencing, accounting for the fact that some readsmay be counted twice. TPM may be calculated by performing the same stepsbut in a different order. First creating a reads per kilobase (RPK) bydividing read counts by the length of each gene, creating the scalingfactor, and then dividing the RPK by the scaling factor to create theTPM.

Other normalization methods may be applied as well, such as one or moreof the RNA normalization methods disclosed in U.S. patent applicationSer. No. 16/581,706, titled “Methods of Normalizing and Correcting RNAExpression Data,” filed Sep. 24, 2019, the entire disclosure of which ishereby expressly incorporated by reference herein. Normalized, encodedread counts may be generated by first normalizing the RNA readsaccording to any of the above methods, and then passing the normalizedread counts to an encoder or a dimensionality reducer, such as anautoencoder.

In one example, an autoencoder may reduce the dimensionality from20,000+transcriptomes to 100 encoded features, creatively named:rna_embedding-z_1 through rna_embedding-z_100. In one example, RNArelated features for each transcriptome are generated from a sequencingof a patient's tumor. The number of encoded features may be any numberwhere identifying the optimal number may include performing encoding foreach of 2-9999 total number of encoded features, calculating aperformance metric of each, and selecting the number of encoded featuresto be the number with the highest performance metric. A performancemetric may include the accuracy of predictions made from the model usingeach total number of encoded features. Raw read counts may be between 0reads and tens of thousands of reads. Normalization of the raw readcounts from sequencing may convert the raw read scores to a valuebetween from −0.5 to 0.5 where 0 represents the mean, or a normalexpression value and −0.5 is lowest expression and 0.5 is highestexpression. The normalized value may represent the number of standarddeviations the raw read was from the normal reads expected in a patientsuch that −0.5 represents a high standard deviation below normal and 0.5represents a high standard deviation above normal. In one example, RNAmay be calculated on a gene or transcriptome basis where variants arenot included. In another example, variants may be included, similar toDNA above.

Encoding normalize RNA reads may include generating a standardpopulation finding or autoencoding. In one example, autoencoding mayinclude utilizing a variational autoencoder, such as Beta-VAE or TC-VAE,or dimensionality reducers, such as SVD, PCA, or UMap. Outputs from anencoder, autoencoder, or dimensionality reducer may be presented as amatrix, where each row is for each patient, and each column is a normaldistributed variable which may be interpreted as a ratio of patient'smakeup in each population, such as values −0.25 to 0.25 or a standarddeviation of 1, centered at 0. A patient's vector of deviations fromnormal may be interpreted to identify the makeup of the patientaccording to each population identified in the respective encoder. Thematrix of normalized, encoded values may be supplied to a model forprediction of metastasis without additional alterations.

Each of the models, raw RNA reads, normalized RNA reads, and normalized,encoded RNA reads may have differing operating characteristics,including speed and accuracy. For example, given the substantial reduceddimensionality from normalized, encoded RNA reads, one may expect thesystem to greatly improve processing speed at the cost of some degree ofaccuracy; however, the resulting ROC AUC may be approximately 0.60 whichis greater than that of processing DNA features only.

A feature set for clinical data only may include: age_at_event (assigneda value equal to the patient's age), age_group {00 to 09, 10 to 19, 100to 109, 110 to 119, 20 to 29, 30 to 39, 40 to 49, 50 to 59, 60 to 69, 70to 79, 80 to 89, 90 to 99, Unknown} (assigned a binary, yes/norepresentation identifying which group the patient's age falls into),days_since_first:TumorFinding:histology {acinar_cell_carcinoma,adenocarcinoma, carcinoma, infiltrating_duct_carcinoma,lobular_carcinoma, malignant_neoplasm,_primary, mucinous_adenocarcinoma,neuroendocrine_carcinoma, non_small_cell_carcinoma, otherGroup,small_cell_carcinoma, small_cell_neuroendocrine_carcinoma,squamous_cell_carcinoma,_no_icd_o_subtype, transitional_cell_carcinoma}(assigned a value equal to the number of days),days_since_first:_TumorFinding:histopath_grade {grade_1(well_differentiated), grade_2_(moderately_differentiated),grade_3_(poorly_differentiated), grade_4_(undifferentiated), high_grade}(assigned a value equal to the number of days),days_since_first:TumorFinding:stage {m0, m1, mx, n0, n1, n2, n3, nx,pn0, pn1, pn2, pnx, stage_1, stage_2, stage_3, stage_4, pt1, pt2, pt3,pt4, t0, t1, t2, t3, t4, tx} (assigned a value equal to the number ofdays), days since first:cancer {breast, cervix_uteri, colon,head_and_neck, kidney, lung,lymphoid,_hemopoietic_and/or_related_tissue, otherGroup, ovary,pancreas, prostate, respiratory_tract, skin, skin_of_trunk,soft_tissues, stomach, tongue, unknown_site, urinary_bladder} (assigneda value equal to the number of days), days_since_last:comorbidity{Abnormal_findings_on_diagnostic_imaging_of_breast,Administration_of_antineoplastic_agent, Anemia, Dehydration,Disorder_of_bone, Disorder_of_breast, Dyspnea, Essential_hypertension,Fatigue, Imaging_of_thorax_abnormal, Immunization_advised,Long_term_current_use_of_drug_therapy, Osteoporosis,Past_history_of_procedure, Pedal_cycle_accident,Screening_for_malignant_neoplasm_of_breast,chronic_obstructive_lung_disease, otherGroup, type_2_diabetes_mellitus,type_2_diabetes_mellitus_without_complication} (assigned a value equalto the number of days), gender {Missing, female, male} (assigned abinary, yes/no representation identifying gender of patient), and race{Missing, african race, american indian or alaska native, asian, pacificislander, black or african american, Caucasian or white, hispanic,native hawaiian or other pacific islander, not hispanic or latino, otherrace, unknown or unknown racial group} (assigned a binary, yes/norepresentation identifying race of patient).

Clinical data features may be assigned weights manually when setting upthe model for metastatic location prediction, may be assigned weightsautomatically via an external weighting model, or assigned weightsautomatically via model itself through a process called stacking.

The resulting ROC AUC may be approximately 0.67 which is greater thanthat of processing DNA features only and RNA features only.

Combining all of the above input feature sets together from the DNAmodel, RNA model, and Clinical data model above results in an ROC AUC ofapproximately 0.70 which is greater than any of the models individually.

In another example, an RNA normalized model using TPM normalization maybe trained on breast cancer patients to predict metastasis to lung,brain, liver, or other organs and have an ROC AUC of approximately 0.92,which is greater than any of the previous models. Further optimizationsmay be pursued by adding in the additional feature sets.

FIG. 14 is an illustration of an example machine of a computer system1400 within which a set of instructions, for causing the machine toperform any one or more of the methodologies discussed herein, may beexecuted. In some implementations, the machine may be connected (such asnetworked) to other machines in a LAN, an intranet, an extranet, and/orthe Internet.

The machine may operate in the capacity of a server or a client machinein client-server network environment, as a peer machine in apeer-to-peer (or distributed) network environment, or as a server or aclient machine in a cloud computing infrastructure or environment. Themachine may be a personal computer (PC), a tablet PC, a set-top box(STB), a Personal Digital Assistant (PDA), a cellular telephone, a webappliance, a server, a network router, a switch or bridge, or anymachine capable of executing a set of instructions (sequential orotherwise) that specify actions to be taken by that machine. Further,while a single machine is illustrated, the term “machine” shall also betaken to include any collection of machines that individually or jointlyexecute a set (or multiple sets) of instructions to perform any one ormore of the methodologies discussed herein.

The computer system 1400 includes a processing device 1402, a mainmemory 1404 (such as read-only memory (ROM), flash memory, dynamicrandom access memory (DRAM) such as synchronous DRAM (SDRAM) or DRAM,etc.), a static memory 1406 (such as flash memory, static random accessmemory (SRAM), etc.), and a data storage device 1418, which communicatewith each other via a bus 1430.

Processing device 1402 represents one or more general-purpose processingdevices such as a microprocessor, a central processing unit, or thelike. More particularly, the processing device may be complexinstruction set computing (CISC) microprocessor, reduced instruction setcomputing (RISC) microprocessor, very long instruction word (VLIW)microprocessor, or processor implementing other instruction sets, orprocessors implementing a combination of instruction sets. Processingdevice 1402 may also be one or more special-purpose processing devicessuch as an application specific integrated circuit (ASIC), a fieldprogrammable gate array (FPGA), a digital signal processor (DSP),network processor, or the like. The processing device 1402 is configuredto execute instructions 1422 for performing the operations and stepsdiscussed herein.

The computer system 1400 may further include a network interface device1408 for connecting to the LAN, intranet, internet, and/or the extranet.The computer system 1400 also may include a video display unit 1410(such as a liquid crystal display (LCD) or a cathode ray tube (CRT)), analphanumeric input device 1412 (such as a keyboard), a cursor controldevice (such as, e.g., a mouse, joystick, or another control device,including a combination device), a signal generation device 1416 (suchas, e.g., a speaker), and a graphic processing unit 1424 (such as, e.g.,a graphics card).

The data storage device 1418 may be a machine-readable storage medium1428 (also known as a computer-readable medium) on which is stored oneor more sets of instructions or software 1422 embodying any one or moreof the methodologies or functions described herein. The instructions1422 may also reside, completely or at least partially, within the mainmemory 1404 and/or within the processing device 1402 during executionthereof by the computer system 1400, the main memory 1404 and theprocessing device 1402 also constituting machine-readable storage media.

In one implementation, the instructions 1422 include instructions for aprediction engine (such as the prediction engine 100 of FIGS. 1-3)and/or a software library containing methods that function as aprediction engine. The instructions 1422 may further includeinstructions for a feature selector 200 and generator 300 and objectivemodules 140. While the machine-readable storage medium 1428 is shown inan example implementation to be a single medium, the term“machine-readable storage medium” should be taken to include a singlemedium or multiple media (such as a centralized or distributed database,and/or associated caches and servers) that store the one or more sets ofinstructions. The term “machine-readable storage medium” shall also betaken to include any medium that is capable of storing or encoding a setof instructions for execution by the machine and that cause the machineto perform any one or more of the methodologies of the presentdisclosure. The term “machine-readable storage medium” shall accordinglybe taken to include, but not be limited to, solid-state memories,optical media and magnetic media. The term “machine-readable storagemedium” shall accordingly exclude transitory storage mediums such assignals unless otherwise specified by identifying the machine-readablestorage medium as a transitory storage medium or transitorymachine-readable storage medium.

In another implementation, a virtual machine 1440 may include a modulefor executing instructions for a feature selector 200 and generator 300and objective modules 140. In computing, a virtual machine (VM) is anemulation of a computer system. Virtual machines are based on computerarchitectures and provide functionality of a physical computer. Theirimplementations may involve specialized hardware, software, or acombination of hardware and software.

Artificial Intelligence Engine Training Pipeline

An exemplary AIE training pipeline may read in a configuration file(such as a JSON) with a number of operating parameters identified. Someparameters may be required while other parameters may be optional.

A pipeline may identify that one or more cohort files may be referencedfor patient data such as a collection of metastatic site location data,diagnosis and metastasis sites location data, or optional extraevaluation sets. The pipeline may also load one or more patient cohortfiles containing information about patient metastasis details, includingthe date and occurrence from a diagnosis site to a metastasis site. Theinformation may provide an indication, such as the date, or number ofdays since a patient metastasized to a site of interest. For a model ofidentifying metastasis from and/or to a specific organ, the informationmay include an indication that a patient metastasized and the time frombiopsy collection to the metastasis. In another example, if there's noindication that the patient metastasized, an indicator may exist thatthere was no observed metastasis and the time from biopsy collection tothe last time that a record exists which evaluated the patient'sdiagnosis (and still showed no metastasis) to generate a detailedmetastasis record for each patient.

The pipeline may identify which feature set(s) are specified and queueup which feature set files for each patient may be loaded in order toaccess and use any relevant features. For example, if it specified thatthe pipeline is to train on a “staging” feature set, the pipeline mayload a “Clinical” feature file, and subset all clinical data down to anystaging features. If it is specified that the pipeline should use CellTexture features, the pipeline may load an Imaging feature set andsubset all imaging data down to any Cell Texture features. The pipelinemay select from any of the patient features disclosed herein and furthermay also join the feature sets from multiple relevant targets into acombined training feature set.

The pipeline may identify an upfront preprocessing function specified inthe configuration file to preprocess the combined training feature setusing the identified preprocessing. In one example, a preprocessingfunction may include one-hot-encoding of categorical features,normalizing features (e.g., condensing separate feature entries forrelated features, such as pathologic N stage and clinical N stage, wherecondensing may include identifying the maximum of those two columns asthe normalized feature), removing uninformative features (e.g., featuresthat just indicate if a field is missing, such as “gender-missing”,“race-missing”, or other status-unknown entries), removing featuresknown to be misleading or problematic (e.g., sequencing normalizationread-throughs), drop features with no variance, imputing missing valuesfrom other data (e.g., when the imputation is reliable), or otherpreprocessing methods.

The pipeline may identify a number of folds for training and subsetwhich features will be used per collection of training set folds. In oneexample, the identification of the number of folds and subsetting offeatures is based upon the combination of inline preprocessing methodand feature selection method. In one example, a total of 5 folds may beselected, [0,1,2,3,4], one (e.g., fold 4) is kept as the hold out set,and the remaining 4 are used in training. Therefore, training sets maybe identified for 5 total folds, including in one example:

[0,1,2] which will be used to generate predictions for fold 3

[0,1,3] which will be used to generate predictions for fold 2

[0,2,3] which will be used to generate predictions for fold 1

[1,2,3] which will be used to generate predictions for fold 0

[0,1,2,3] which will be used to generate predictions for the test set(fold 4)

Generating the combined feature sets for each fold, or the 5 differenttraining sets defined above, may include, in one example, the followingsequence of events:

-   -   1) Run the specified in-line preprocessing method using one or        more of:        -   a) Transformations to zero-center features (e.g., z-scoring)        -   b) Transformations to scale features relative to the maximum            observed value        -   c) Dimensionality reduction (e.g., PCA)        -   d) Subsetting to the top X correlated features to the target            (where the target can be defined as the binary target, time            until metastasis for only patients who metastasize, the log            of that duration, or another format).    -   2) Run the specified feature selection method on the in-line        processed data using one or more of:        -   a) A custom feature selection approach using Lasso modeling            such as by re-sampling with replacement a number of times            (e.g., 100 times) from each training subset (e.g., folds            [0,1,2]), and fitting a lasso model on each bootstrap. each            lasso model, storing the features that were used in the            model, and the associated regression coefficients. The            features that are most important for any given training            subset are the ones that appear in the most bootstrapped            models, have high (in magnitude) coefficients, and have            stable coefficients across models (in terms of the sign of            the coefficient). In one example, identifying feature            selection sets may include selecting the features that are            occur in more than a minimum percentage (e.g., 50%) of            bootstraps, have the same sign of their coefficient at least            some minimum percent (e.g., 90%) of the time that they are            used.        -   b) A custom recursive feature elimination framework, such as            by running a model on all features (or subset of features if            defined in the inline preprocessing method), dropping the            bottom (e.g., 10%) of features as ranked by their model            coefficients, and repeating the feature elimination until a            threshold number of features is met (e.g., 10, 50, 200.            5000), At each step of the process, each feature's rank is            stored. At the end (once only Y features remain), the            original combined feature set may be ranked, each by their            average rank from this process, and only the top Z            (e.g., 40) features may be selected as features for that            training subset. Recursive feature elimination may include            logistic regression, cox proportional hazards, early            stopping, ranking/selection methods, and others.    -   3) Storing the selected features for use in each fold.    -   4) Optimizing hyperparameters, such as a gridsearch for the set        of hyperparameters using one or more of:        -   a) logistic regression            -   i) identifying regularization strength the range 100,                10, 1, 0.1, 0.01, and 0.001.        -   b) cox proportional hazards        -   c) random forest            -   i) number of trees, such as 20, 40, 60, or 80.            -   ii) maximum depth of each tree, such as 2, 3, 4, 10, 20,                100, branches.            -   iii) minimum samples per leaf, such as 5, 6, 7, 10, 100            -   iv) The metric to optimize for. For example, ROC AUC or                concordance index.    -   5) The pipeline may cycle through all the training subsets, for        example, the four training subsets [0,1,2], [0,1,3], [0,2,3],        and [1,2,3]), using the normalized and selected feature sets.        Then, for each possible hyperparameter space, fitting the        identified model on the training subset, predict on the        remaining training fold, and storing the resulting the metric        which is being optimized for (e.g., ROC AIX, concordance index)        on the held out fold. Each search space (e.g., the combined        training subset metric results) may then be associated with 4        out of fold metrics. The hyperparameter set that leads to the        best average metric (averaged across those 4 out of fold        estimates) is stored as the optimal hyperparameters of the        model.    -   6) The pipeline may generate the final prediction on the test        fold using the combined feature selected subset from each fold        and the model identified with the optimal hyperparameters for        the model to predict the output on the test fold and store the        predictions.    -   7) Identify and store features which were most important in        driving the predictions, based on the feature selection        method(s) selected using one or more of:        -   a) Spearman correlation between the feature and predictions,        -   b) Pearson correlation between the feature and predictions,        -   c) Kendall correlation between the feature and predictions,        -   d) Custom subset aware feature effect correlation            identification,        -   e) Nulling-out method where all values of a feature may be            set to 0, and compute the mean absolute deviation in            resulting probabilities based on the rest of the features.    -   8) The prediction results may be stored in one or more patient        information databases and all stored metrics may be saved to the        pipeline as a model for predicting future metastatic site        occurrence in a new patient.

Prediction of Colorectal Metastasis Based on RNA

In one embodiment, the artificial intelligence engine may implement anRNA-based predictive algorithm to predict how likely a colorectal cancerpatient is to develop a metastasis.

Training a model may include receiving sequencing results from 120,000patients having a primary diagnosis of colorectal cancer wherein 10,000patients have at least one tumor RNA sequencing result from a biopsysite of the colon. These 10,000 patients have been filtered from the120,000 larger patient dataset to include only patients who are stage IIor stage III only, excluding patients who have already metastasized andare stage IV at the time of the biopsy for the tumor tissue and in theevent a patient may have multiple sequencing results, selecting thesequencing results that were biopsied from the colon at Stage III orStage IV.

A training model pipeline may be initialized with the followingparameters:

-   -   1) An input training dataset having all features for all        patients, including TPM normalized, low pass, whole        transcriptome RNA sequencing results for 120,000 patients.    -   2) A diagnosis site of interest of colorectal cancer. So that        only patients whose tumor started in the colon are selected for        model training.    -   3) A metastasis site of interest of any site. So that patients        who metastasized after sequencing are included in the training        set when any metastasis location is identified.    -   4) A horizon window of 24 months. So that only patients having        records that extend through the period of the trained model        prediction horizon are included. Patients who metastasized        before the window closes may be included even if records fail to        be present for the entire horizon.    -   5) A sample restriction of no Cancer Staging of I or IV at time        of sequencing. So that patients whose sequencing results were        generated from Stage I or Stage IV are excluded.    -   6) A data requirement of none. So that no data inclusion        requirements are imposed on the patient set.    -   7) A modeling framework of logistic regression.    -   8) A hyperparameter space of regularization strength:        C=[100,10,1,0.1,0.01,0.001]. So that matching hyperparameters        for the logistic regression framework are selected.    -   9) A hyperparameter tuning optimization metric of ROC AUC    -   10) A preprocessing option of RNA TPM.    -   11) An inline preprocessing of transforms.    -   12) A feature selection method of passthrough.    -   13) A feature importance method of spearman correlation.

A resulting pipeline run may generate a set of 5 folds, 4 training andone testing. The pipeline using a combined feature set of RNA TPMfeatures with Lasso feature selection method may result in 359 featuresfor training subset [0,1,2], 361 features for training subset [0,1,3],347 features for training subset [0,2,3], 331 features for trainingsubset [1,2,3], and 404 features for training subset [0,1,2,3]. In oneexample, all RNA TPM features may include over 20,000 genes or over198,000 transcripts which may be mapped to genes. A complete feature setincluding all of the RNA TPM features which are selected during featureselect may be optimized to include only 1030 genes and transcripts:A3GALT2, A4GNT, AADAC, AADACL2, ABCC1, ABTB2, ACAD9, ACHE, ACOT1, ACOT7,ACPL2, ACTR10, ACTR3B, ACVR2B, ADAM30, ADAMTS6, ADAP1, ADAT2, ADHFE1,ADIPOR1, ADORA2B, ADPRHL1, ADPRM, AFAP1, AGAP7, AGAP9, AGPAT6, AGPS,AGR2, AGTR2, AGXT, AHNAK, AHSA2, AIF1, AIM1L, AK1, AK2, AK3, AKR1B1,AKR1B15, AKR1C4, AKR7A3, ALDH1A3, ALG10, ALG14, ALG1L, ALKBH6, ALS2,AMIGO3, AMMECR1, ANAPC7, ANP32E, AOAH, APPL1, AQP8, ARHGAP1, ARHGDIB,ARHGEF35, ARMC4, ARSD, ASAH2, ASB8, ASH2L, ASMT, ASPG, ASPH, ASPRV1,ATF4, ATP13A1, ATP13A2, ATP5A1, ATP6V1C2, ATP6V1E2, ATRX, ATXN7L3B,AVEN, B2M, BAG2, BARD1, BBS9, BCL2L2, BEST4, BET1L, BHLHB9, BRMS1L,BTN2A1, BZRAP1, C10orf111, C10orf128, C11orf86, C12orf73, C14orf93,C15orf38, C16orf3, C16orf80, C17orf89, C19orf35, C19orf44, C1orf21,C1orf227, C1orf229, C1orf51, C1orf95, C1QL3, C1QTNF4, C1QTNF8, C21orf49,C22orf42, C2orf27B, C2orf48, C2orf72, C2orf91, C3, C3orf17, C3orf84,C4orf22, C4orf45, C4orf6, C5orf20, C5orf64, C6orf47, C6orf48, C6orf52,C6orf99, C7orf66, C7orf71, CBG, C8orf34, C8orf34-AS1, C9orf129,C9orf131, C9orf156, C9orf24, C9orf50, C9orf64, C9orf91, CA1, CABP2,CABP4, CACNA1E, CACNG4, CAPN2, CARTPT, CBR3, CCDC125, CCDC141, CCDC144A,CCDC17, CCDC175, CCDC183, CCDC25, CCDC78, CCDC81, CCDC89, CCIN, CCNA1,CCNE1, CCR6, CD151, CD200R1, CD247, CD28, CD4, CD9, CDA, CDIPT, CDK20,CDKN2AIP, CDNF, CEACAM16, CEACAM18, CEACAM3, CEACAM5, CEP76, CEP85,CFAP206, CHD5, CHKB, CHMP7, CKMT1B, CLDN10, CLDN22, CLDN23, CLDN34,CLEC18B, CLECL1, CLIC2, CLN8, CLP1, CLPB, CMSS1, CNBD1, CNOT6L, COA4,COCH, COMMD7, COMMD9, COPG2, COQ5, COX6A2, CPLX4, CRABP1, CRHR2, CRTC3,CRX, CRYBB2, CSK, CSNK2B, CTAGE6, CTBP2, CTTNBP2NL, CXorf38, CXXC1,CYLC2, CYorf17, CYP11B2, CYP27B1, CYP2C19, CYP2C8, CYP4A22, DAB1,DAZAP2, DAZL, DBF4, DCAF7, DCBLD1, DCLRE1B, DDHD1, DDHD2, DDO, DDR1,DDX4, DDX54, DDX56, DEFB127, DEFB134, DEGS2, DEXI, DGAT1, DGCR14, DHDDS,DHPS, DHRS4L2, DHRS7B, DIRC1, DNAH1, DNAH5, DNAJC27, DNAJC8, DND1,DOCK2, DPM2, DPPA3, DPY19L2, DRD1, DUS4L, DUSP27, DYNLL2, DYRK2, DYRK4,EBNA1BP2, ECHDC1, EDDM3A, EDDM3B, EFHC2, EFHD2, EIF3CL, ELANE, ELMOD1,EMID1, ENPP2, ENPP7, ENSG00000171282, ENSG00000173366, ENSG00000187461,ENSG00000187811, ENSG00000197665, ENSG00000203546, ENSG00000220032,ENSG00000241690, ENSG00000250232, ENSG00000251012, ENSG00000251606,ENSG00000254673, ENSG00000254943, ENSG00000254979, ENSG00000256100,ENSG00000258365, ENSG00000258881, ENSG00000259316, ENSG00000259649,ENSG00000260371, ENSG00000260836, ENSG00000260861, ENSG00000260869,ENSG00000266202, ENSG00000266956, ENSG00000267140, ENSG00000267157,ENSG00000267964, ENSG00000268170, ENSG00000268643, ENSG00000268702,ENSG00000268714, ENSG00000268950, ENSG00000269711, ENSG00000269846,ENSG00000272195, ENSG00000272617, ENSG00000272762, ENSG00000272822,ENSG00000272896, ENSG00000273217, ENSG00000273266, ENTPD3, EPHB1,EPM2AIP1, EPX, EQTN, ERCC6, ETNK1, ETS1, ETV2, ETV6, EVPLL, EXOC3L4,EYA1, FABP9, FAHD2B, FAM132B, FAM151A, FAM166A, FAM183A, FAM19A4,FAM219B, FAM21B, FAM228A, FAM24B, FAM26D, FAM71C, FAM72B, FAM83F,FAM98A, FBXL6, FBXO47, FBXW12, FDFT1, FDXR, FEZF2, FHIT, FKBP2, FKBP6,FKBPL, FNDCS, FOXD1, FRRS1L, FSCB, FSHR, FUS, FXR1, FZD10, G6PD, GABRR2,GAL3ST1, GAR1, GATSL1, GATSL3, GCSAML, GDF15, GGCT, GGNBP2, GGT1,GIMAP2, GLS2, GMEB1, GNRHR, GOLGA6L1, GOLGA7, GOLGA8O, GOLM1, GPATCH1,GPATCH2L, GPC2, GPR112, GPR115, GPR143, GPR152, GPR19, GPR55, GRAPL,GRIP1, GRK7, GRM2, GSG1L, GSPT2, GSTT1, GTF2A1L, GTF2H2C, GTF3A, GTF3C6,GUCA2B, GVQW3, GZF1, H2AFB1, HACE1, HACL1, HAL, HBD, HBZ, HCN1, HCRTR1,HDDC3, HEBP1, HINT3, HIRA, HIRIP3, HLA-DMB, HLA-DPB1, HLA-DQB1,HLA-DRB1, HLA-G, HMX3, HNF4G, HNRNPA2B1, HNRNPCL1, HNRNPR, HOMEZ,HORMAD1, HORMAD2, HOXA1, HOXA11, HOXA13, HOXA3, HR, HRNR, HS6ST2, HSCB,HSD17B13, HSD17B8, HSPA8, HTATSF1, HTR5A-AS1, HTR6, ID2, IDH2, IFIT1,IFNA2, IFRD2, IFT172, IGF2BP2, IGFN1, IGSF3, IL12A, IL12B, IL17RB,IL18R1, IL2, IL4, IL9, INPP5A, INSR, IPO4, IPO9, IQCF6, IQCK, IREB2,ISCA1, IVD, KCNA10, KCNS3, KCTD18, KDM5C, KIAA1211, KIAA1467, KIAA1683,KIF21B, KIF26A, KIF5A, KLF11, KLHDC10, KLHL10, KLHL40, KLK2, KLRD1,KRT12, KRT24, KRT40, L3MBTL4, LANCL1, LDHD, LENG9, LHCGR, LILRB2,LILRB4, LIMS1, LIN7A, LMBR1L, LONRF3, LRAT, LRP8, LRR1, LRRC57, LRRC59,LRRFIP1, LSM14A, LTBP2, LUZP1, LUZP2, LY6G5C, LY6G6F, LY86, LYG1, M6PR,MAGEB5, MAGIX, MAL2, MAN1A1, MAP4K2, MAPK15, MAPKBP1, MARK3, MASP2,MC2R, MCOLN3, MDGA2, ME3, MECR, MED9, MESDC2, METTL11B, MFSD2B, MFSD9,MIB1, MICALL2, MINOS1, MKS1, MLF2, MLLT4, MLPH, MMP17, MOB1B, MPL, MPP3,MPP5, MPP7, MPZL1, MRPL38, MS4A8, MSH5, MSL3, MT1F, MTCP1, MTDH, MTUS1,MTX3, MX1, MYBPC3, MYCBP, MYH15, MYH2, MYLK4, MYO1A, MYO9A, N4BP1,NDUFA3, NDUFC2, NEUROD1, NFRKB, NFXL1, NGRN, NHLH2, NKAIN2, NKAPL,NLRP9, NME2, NMNAT1, NOL7, NOXRED1, NPC1L1, NPHS2, NPIPA8, NPIPB15,NPRL3, NPTX2, NPY4R, NPY5R, NR2C2, NRG4, NRTN, NT5DC4, NTSR1, NTSR2,NUCB1, NUDC, NUDT8, NUTM2B, NWD1, OAZ2, OBP2A, ODAM, OLFM4, OPCML,OPLAH, OPTN, OR10A2, OR10A5, OR10P1, OR13F1, OR1E1, OR2AE1, OR2AK2,OR2H1, OR2M4, OR2T12, OR2T2, OR2T27, OR2Y1, OR4F15, OR5111, OR52B6,OR52D1, OR5AC2, OR5H6, OR5K3, OR6A2, OR6B3, OR6C76, OR7D4, OR7G3, ORC6,OSBPL3, OTOP2, OXER1, P2RY8, P4HB, PABPC1, PABPC3, PACS1, PACSIN2,PADI3, PALM3, PANK4, PAPOLB, PAQR6, PARD6G-AS1, PARL, PARP11, PARVB,PAX8, PCCA, PCDH19, PCDHB5, PCDHGA2, PCDHGB1, PCGF2, PCNP, PDE10A,PDE8B, PDZD8, PECR, PEX2, PFN3, PHC1, PHF20L1, PHTF1, PIH1D2, PIK3C2G,PINX1, PIWIL1, PIWIL3, PKNOX1, PLEKHO1, PLK5, PLSCR5, PLXNA2, PNMA2,PNP, PODXL, POGLUT1, POLD3, POLG2, POLR3F, POM121, POM121C, PPAPDC1B,PPIAL4A, PPIAL4B, PPIAL4G, PPM1J, PPP1CA, PPP1R1C, PPP1R36, PPP2R2D,PPP2R5B, PPY, PRAIVIEF12, PRAIVIEF21, PRB1, PRDM10, PRDM12, PRKG2,PROSC, PRPF38B, PRPF39, PRPH, PRR4, PRRT2, PRSS21, PRSS46, PRSS53,PSMD9, PSMF1, PSPC1, PTDSS2, PTER, PTGES3L, PTPN20B, PTPRG, PTPRQ,PVRL3, PXDNL, QDPR, RAB11FIP1, RAB1B, RAB40AL, RABGGTA, RAD23A, RAD50,RAET1E, RANBP1, RAPH1, RASEF, RASGEF1B, RBCK1, RBM11, RBM23, RBM34,RCAN3, RCVRN, RD3L, RDH8, REG1B, RELN, RFTN2, RFX3, RGS14, RGS6, RHOXF1,RHPN2, RIN3, RIPK1, RIPPLY1, RLN1, RNASE11, RNASE9, RNF214, RNF25,RNF31, ROBO3, ROS1, RPA4, RPAIN, RPEL1, RPF1, RPGR, RPL36, RPL39L, RPRM,RPS3, RPS6KB2, RRS1, RSPH4A, RTN4R, RTP1, RUVBL2, SAE1, SAP25, SAPCD1,SBSPON, SCIN, SCN4A, SCP2D1, SCPEP1, SDCBP2, SEC23IP, SECISBP2, SELENOF,SEMA4G, SEPW1, SERINC3, SERP2, SETMAR, SF3B2, SFN, SFTPD, SGCZ, SH3BGR,SHISA4, SIK2, SIM2, SIRPD, SLC10A6, SLC11A1, SLC13A5, SLC22A31, SLC22A7,SLC24A2, SLC25A2, SLC25A26, SLC25A47, SLC25A53, SLC2Al2, SLC34A3,SLC35E1, SLC35G5, SLC3A2, SLC44A5, SLC4A9, SLC6Al2, SLC6A8, SLC7A7,SLC9A7, SMAD2, SMAD5, SMCR8, SMIM11, SMIM13, SMIM5, SMR3A, SNAP29,SNIP1, SNRPA1, SNTG1, SNX2, SNX31, SNX32, SOCS2, SOX8, SPACA5B, SPAG1,SPAG4, SPATA22, SPATA3, SPATA31D1, SPDYC, SPOCK2, SPPL3, SPRED1, SPRED2,SPTBN1, SPTLC3, SRSF12, SRSF3, SSTR5, ST6GAL2, ST8SIA3, ST8SIA6, STAG3,STMN4, STMND1, STPG2, STRN3, SULT1A2, SUMF1, SUMF2, SUPT5H, SUV420H2,SYNDIG1L, SYT17, SYT2, TAAR8, TAC3, TACC1, TACC2, TACR1, TAF12, TAMM41,TAP2, TAS2R31, TAS2R39, TAS2R43, TAS2R46, TAS2R8, TBC1D2B, TBC1D30,TBCCD1, TBRG1, TCAP, TCEB3CL2, TCN2, TCTA, TCTN1, TDRD9, TEDDM1, TEPP,TERF1, TEX14, TEX30, TEX33, TEX40, TFIP11, TGM2, THAP8, TIMM22, TIMMDC1,TINAGL1, TM6SF2, TM7SF2, TMC4, TMED2, TMEM107, TMEM125, TMEM132C,TMEM151B, TMEM155, TMEM170B, TMEM178B, TMEM180, TMEM206, TMEM220,TMEM248, TMEM249, TMEM33, TMEM54, TMEM55B, TMPRSS11A, TMPRSS11D, TMSB4Y,TNNI3, TNNI3K, TOE1, TP53I11, TP53TG3D, TPPP, TPPP3, TPR, TRDMT1,TRIM16, TRIM16L, TRIM24, TRIM36, TRIM46, TRIM6, TRIM64C, TRMT1, TRMT2A,TRPC3, TRPC6, TRPC7, TSC1, TSEN34, TSNAXIP1, TSSC1, TSSK4, TSTA3,TTC21A, TTC29, TTC6, TTYH2, TUBA3E, TUBAE, TUBB8, TXK, TXLNB, TXN2,U2AF1L4, UBE2E3, UBE2G2, UBE2V1, UBFD1, UBQLNL, UBXN10, UCN, UGT3A1,ULK4, UMPS, UNC13B, UPRT, UQCC1, URGCP, URI1, USP41, VANGL1, VANGL2,VAPB, VAX2, VBP1, VCPKMT, VIMP, VPS11, VTI1B, VTN, WASF3, WBSCR16,WDR63, WDR74, WDR91, YBX2, YDJC, YKT6, ZBBX, ZBTB7B, ZC2HC1B, ZC4H2,ZCCHC17, ZDHHC19, ZDHHC5, ZFAND2A, ZIC5, ZNF114, ZNF266, ZNF284, ZNF317,ZNF329, ZNF460, ZNF554, ZNF561, ZNF576, ZNF606, ZNF662, ZNF692, ZNF705B,ZNF705G, ZNF98, ZP3, ZSCAN20, ZSCAN30, ZSWIM2, and ZSWIM7.

A lasso model input feature set may be further limited to only genes andtranscripts which are most impactful in the prediction, including 404genes and transcripts: A4GNT, ABTB2, ACHE, ACOT1, ACOT7, ACTR10, ACTR3B,ACVR2B, ADHFE1, ADIPOR1, ADPRHL1, AGPAT6, AGTR2, AGXT, AHNAK, AIM1L,AK2, AKR1B1, AKR1C4, ALDH1A3, ALG14, ALG1L, ALKBH6, AMIGO3, AMMECR1,ANP32E, AOAH, ARHGDIB, ARMC4, ARSD, ASAH2, ASB8, ASPG, ASPH, ATP13A1,ATP13A2, ATP6V1E2, ATRX, ATXN7L3B, BET1L, BTN2A1, BZRAP1, C12orf73,C14orf93, C15orf38, C16orf3, C19orf35, C1orf227, C1orf229, C1QL3,C2orf27B, C2orf48, C2orf91, C3orf84, C4orf45, C5orf20, C6orf47, C6orf52,CBG, C8orf34, C9orf129, C9orf156, C9orf91, CABP4, CACNG4, CBR3, CCDC125,CCDC141, CCDC144A, CCDC183, CCDC25, CCDC78, CCDC89, CD151, CD200R1,CD247, CDA, CEACAM18, CEACAMS, CKMT1B, CLDN10, CLDN34, CLEC18B, CLPB,COCH, COMMD7, COMMD9, COPG2, COQS, COX6A2, CPLX4, CTBP2, CXorf38, DAB1,DBF4, DCBLD1, DDHD1, DEFB134, DEGS2, DHDDS, DHRS4L2, DIRC1, DPM2,DUSP27, DYRK4, ECHDC1, EDDM3A, ELMOD1, EMID1, ENSG00000187461,ENSG00000187811, ENSG00000197665, ENSG00000203546, ENSG00000250232,ENSG00000254673, ENSG00000256100, ENSG00000260836, ENSG00000266956,ENSG00000267140, ENSG00000267157, ENSG00000268714, ENSG00000268950,ENSG00000269846, ENSG00000272617, ENSG00000272762, ENSG00000273217,EPHB1, EQTN, ERCC6, ETS1, ETV2, ETV6, EYA1, FABP9, FAHD2B, FAM21B,FAM72B, FAM83F, FBXO47, FDFT1, FEZF2, FHIT, FKBP2, FOXD1, FRRS1L, FSHR,FUS, FXR1, G6PD, GATSL3, GDF15, GGT1, GLS2, GMEB1, GOLGA6L1, GOLGA8O,GOLM1, GPATCH2L, GPR112, GPR115, GPR19, GPR55, GRK7, GSPT2, GTF2A1L,GVQW3, HDDC3, HIRA, HIRIP3, HLA-DMB, HLA-DPB1, HLA-DQB1, HMX3, HORMAD1,HOXA11, HOXA13, HRNR, HS6ST2, HSD17B13, HTR6, IFRD2, IGF2BP2, INPP5A,ISCA1, KCNS3, KIAA1467, KIAA1683, KIF21B, KIF5A, KLHDC10, KLK2, KRT12,KRT40, L3MBTL4, LDHD, LHCGR, LILRB2, LILRB4, LMBR1L, LONRF3, LRAT, LRR1,LRRC57, LRRC59, LUZP1, LY6G5C, M6PR, MAGEBS, MAGIX, MAPKBP1, MARK3,MEDS, METTL11B, MFSD9, MIB1, MICALL2, MKS1, MLPH, MMP17, MPL, MPP3,MPP7, MSHS, MT1F, MTCP1, MTDH, MTUS1, MYBPC3, MYCBP, MYH2, MYLK4, MYO1A,NDUFA3, NFXL1, NGRN, NLRP9, NME2, NMNAT1, NPC1L1, NPRL3, NPY5R, NRG4,NRTN, NUCB1, NUDC, NWD1, OAZ2, OBP2A, OLFM4, OPLAH, OR2AE1, OR4F15,OR52B6, OR5AC2, ORC6, OSBPL3, OXER1, PADI3, PALM3, PANK4, PAPOLB, PAQR6,PARD6G-AS1, PCDH19, PCDHB5, PCDHGA2, PCNP, PDE10A, PDE8B, PEX2, PHF20L1,PIK3C2G, PLK5, PLSCR5, PLXNA2, POLR3F, POM121C, PPIAL4A, PPP1R36,PPP2R2D, PRAMEF21, PRKG2, PROSC, PRR4, PRSS53, PSPC1, PTER, PTPN20B,RAB40AL, RABGGTA, RAD50, RASEF, RASGEF1B, RBCK1, RCAN3, RCVRN, RD3L,RELN, RGS14, RGS6, RIN3, RNASE11, RNF25, ROBO3, RPA4, RPAIN, RPEL1,RPGR, RPL39L, RPS3, SAP25, SAPCD1, SBSPON, SCIN, SDCBP2, SETMAR, SH3BGR,SHISA4, SIRPD, SLC24A2, SLC35E1, SLC7A7, SLC9A7, SMAD5, SMIM11, SNRPA1,SNTG1, SOCS2, SOX8, SPATA22, SPATA3, SPOCK2, SPRED1, SPRED2, SPTBN1,SRSF12, ST8SIA6, STPG2, STRN3, SULT1A2, TAC3, TACC1, TAF12, TAS2R31,TAS2R43, TAS2R46, TBC1D2B, TBC1D30, TBCCD1, TCTN1, TDRD9, TEDDM1, TERF1,TEX40, TFIP11, TIMMDC1, TM6SF2, TMC4, TMEM107, TMEM125, TMEM151B,TMEM155, TMEM180, TMEM206, TMEM249, TNNI3, TPR, TRDMT1, TRIM16L, TRIM36,TRIM6, TRMT1, TRMT2A, TRPC3, TSC1, TSEN34, TSNAXIP1, TSSC1, TTC21A,TTC29, TTYH2, TUBA3E, TXK, U2AF1L4, UBE2V1, UBFD1, UBQLNL, UGT3A1, UMPS,UNC13B, URI1, VAPB, VAX2, VTI1B, WASF3, WBSCR16, YKT6, ZBTB7B, ZCCHC17,ZDHHC19, ZNF266, ZNF284, ZNF705B, and ZNF705G.

In another example, a resulting pipeline run may generate a set of 5folds, 4 training and one testing. The pipeline using a combined featureset of RNA TPM features with recursive feature elimination featureselection method may result in 40 features for training subset [0,1,2],40 features for training subset [0,1,3], 40 features for training subset[0,2,3], 40 features for training subset [1,2,3], and 40 features fortraining subset [0,1,2,3]. In one example, all RNA TPM features mayinclude over 20,000 genes or over 198,000 transcripts which may bemapped to genes. A complete feature set including all of the RNA TPMfeatures which are selected during feature select may be optimized toinclude only 112 genes and transcripts: ALG10, ANKH, ART1, BRK1,C10orf71, C11orf73, C14orf177, C1orf167, CACNG1, CCER1, CCZ1, CHRNA2,CNIH1, CNOT8, COPG2, CRKL, CYLC2, DEFB4B, DLL3, DPPA2, ENSG00000254943,ENSG00000255439, ENSG00000258881, ENSG00000269881, ESD, FAM163A,FAM180B, FEZF2, GRK7, GRM6, GTPBP1, HAT1, HCRTR1, HRK, KCNK4, LAPTM4A,LCE1A, LCE6A, LGALS16, LOR, LPCAT3, LY6L, MAGOHB, MFSD2B, MPZL1,MRGPRX3, MTRNR2L10, NDUFA4, NDUFB1, NDUFC2, NDUFS4, NUP133, OPN1MW,OR10G3, OR10G4, ORM, OR2G6, OR2T34, OR6C76, OR8S1, PABPC1L2A, PIGF, PLG,PON2, PRDM12, PRDX6, RDH8, RHO, RLBP1, ROM1, RPL10A, RPL41, RPS27,RSL24D1, SF3A1, SIM1, SLC17A2, SLC17A6, SLC38A2, SMARCC1, SMDT1, SOX15,SPACA1, SPIN1, SPPL3, SRSF3, SSX2B, STX17, TECRL, TEX261, TEX33,THUMPD1, TM2D1, TMEM167A, TMEM251, TMEM258, TRAPPC10, TRIM49, TRIML1,TRIT1, TRMT2A, TULP1, UBE2E1, UBE2Q1, UNC50, UPRT, VMA21, YPEL5, ZC3H7A,ZNF311, and ZP1.

A recursive feature elimination model input feature set may be furtherlimited to only the top 40 genes and transcripts which are mostimpactful in the prediction, including: CACNG1, CNIH1, CRKL, CYLC2,DEFB4B, ENSG00000258881, ENSG00000269881, ESD, FAM163A, FAM180B, GRM6,HAT1, LAPTM4A, LCE6A, MFSD2B, MPZL1, MTRNR2L10, NDUFA4, NUP133, OR6C76,OR8S1, PABPC1L2A, PIGF, PLG, PRDM12, ROM1, RPL41, RPS27, SLC38A2, SMDT1,SOX15, SPIN1, SRSF3, STX17, THUMPD1, TMEM167A, TMEM258, TRIT1, YPEL5,and ZNF311.

In another embodiment, the lasso and/or recursive feature eliminationfeature selection models above may select a trained model input from 5or more, 10 or more, 25 or more, 50 or more, 75 or more, 125 or more, oranother selected number of features from the feature sets identifiedabove. In one example, embodiments including a selected number offeatures may be used when whole transcriptome results are not available.

In another embodiment, information related to the expression level of agene may be correlated with or predicted from the expression level dataassociated with one or more other genes. When a gene may be predictedfrom one or more other genes, the one or more other genes are defined asproxy genes. In one example, a gene of a trained model may be replacedby its proxies or predicted from the proxies and the prediction fed inits place.

One or more of the final models may be used to predict patientmetastasis from cancers having a diagnosis of colorectal to any othersite/organ.

The methods and systems described above may be implemented as acomponent of various practical applications. For example, a person mayexperience symptoms such as, e.g., unexpected weight loss and a coughthat persists for several weeks. Concerned for their overall wellbeing,they may seek a diagnosis from a physician. The physician may recognizethe person's symptoms as indicative of lung cancer and schedule imagingof the patient's lung with a Computed Tomography (CT) scan of the chest.Imaging results may come back identifying a suspected tumor in theperson's lung. The person, now patient of an oncologist (also called thephysician), may have a biopsy performed which identifies the tumor asmalignant. The physician may then send a biopsy to a pathologist fordiagnosis and to have the tumor sequenced to identify any drivers of thepatient's lung cancer. The pathologist may identify the lung cancer asnon-small cell lung cancer (NSCLC).

In some embodiments, a tumor specimen and blood sample may be sent to anext-generation sequencing laboratory for Tumor-Normal sequencing. TheDNA and RNA may be isolated from the tumor tissue specimen andsequencing may be performed on an Illumina sequencer. As used herein,“Matched Tumor-Normal,” “Tumor-Normal Matched,” and “Tumor-NormalSequencing” defines processing genomic information from a subject'snormal sample, such as saliva, blood, urine, stool, hair, healthytissue, or other collections of cells or fluids from the subject, andgenomic information from a subject's tumor sample, such as smears,biopsies or other collections of cells or fluids from a subject whichcontain tumor tissue, cells, or DNA. DNA and RNA features which havebeen identified from a next generation sequencing (NGS) of a subject'stumor or normal specimen may be cross-referenced to remove genomicmutations and/or variants which appear as part of a subject's germlinefrom the somatic analysis. The use of a somatic and germline datasetleads to substantial improvements in mutation identification and areduction in false positive rates. Tumor-Normal Matched Sequencingprovides a more accurate variant calling due to improved germlinemutation filtering. For example, generating a variant call based atleast in part on the germline and somatic specimen may includeidentifying common mutations and removing them. In such a manner,variant calls from the germline are removed from variant calls from thesomatic as non-driver mutations. A variant call that occurs in both thegermline and the somatic specimen may be presumed to be normal to thepatient and removed from further bioinformatics calculations.

The same procedure may be performed on the blood sample as the normalsequencing so that results from the RNA and DNA results of both tumorand normal sequencing may be analyzed. A sequencer, such as thesequencer generating results for the Tumor-Normal sequencing, maygenerate a FASTQ file having a plurality of reads from the sequencing.After generation of a FASTQ file, the file may be uploaded to acloud-based platform or processed locally. Reads may be aligned to areference genome using paired-end reads to increase the accuracy.Aligned reads may be stored as a BAM file. A bioinformatics pipeline mayreceive the BAM file and identify variant calls, gene mutations,fusions, alterations, copy number states, and other alterations asdescribed above. In an example, the sequencing and subsequent processingof RNA and/or DNA obtained from the patient sample may identify avariant in one of the following genes: kirsten rat sarcoma viraloncogene (KRAS), anaplastic lymphoma kinase receptor (ALK), humanepidermal growth factor receptor 2 (HER2), v-raf murine sarcoma viraloncogene homolog B1 (BRAF), PI3K catalytic protein alpha (PI3KCA), AKT1,MAPK kinase 1 (MAP2K1 or MEK1), or MET, which encodes the hepatocytegrowth factor receptor (HGFR). In one example, mutations may beidentified in the EGFR gene. A report may be generated, summarizing theresults from the bioinformatics pipeline, including clinical trials andtherapies that are determined to be most relevant to the patient'sparticular genome.

As part of the report generation, predictions may be generated by anartificial intelligence engine, such as an engine included in the systemin accordance with the present disclosure and configured to perform thedescribed processes. In one example, a prediction may be generated as tothe likelihood that the patient's lung cancer may metastasize to anotherorgan, such as the brain, liver, or bone. A report, summarizing thefindings from the pathologist and subsequent sequencing may be generatedfor the physician. The report can be used by a physician to consider theprediction provided by the report. For example, the generated predictioncan be that the patient is likely to metastasize to the brain in thenext 12 months. This information may allow making a decision regardingwhether to schedule a surgery for the patient, a combination of surgeryand endobronchial therapy, surgery and radiation therapy, surgery andchemotherapy, or cytotoxic chemotherapy in combination with EGFRtyrosine kinase inhibitors. In one example, the physician may elect forsurgery with increased monitoring afterwards, such as by regularly CTimaging and 18F-FDG positron-emission tomography (PET) scanning. Inanother example, the physician may elect a combination of cytotoxicchemotherapy in combination with EGFR tyrosine kinase inhibitors toaggressively treat the patient. The patient, because of the selectedtherapy, may experience a substantially improved response and outcome totreatment. The patient's NSCLC may go into remission and the patient mayremain progression free until the patient's natural death of old age. Aphysician may schedule regular monitoring through CT imaging or PETscanning. The power of the reporting, including a prediction of thepatient's likelihood to metastasis to another organ, is in allowing thephysician to provide the most expedient, affordable care to the patientby applying the benefits of precision medicine over a one-size fits allcare regimen.

In furtherance of the above patient timeline, generation of thepredictions may be performed in accordance with the method and systemsdisclosed above. The system, such as, e.g., the system 100 forgenerating and modeling predictions of patient objectives (FIG. 1), maygenerate the predictions to be included in the report to the physician.In one example, an artificial intelligence engine or a componentthereof, such as, for example, the metastasis site objective module 146of Objective Modules 140, may generate the prediction from patientrecords.

In an example, the artificial intelligence engine may be initiated for aTarget/Objective of Metastasis from Lung to Brain within 12 months. Amodel 146 b may be trained on labeled RNA feature data. The labeled RNAfeature data may be labeled with information about whether the patientfrom whom the RNA feature data was derived had a form of lung cancerthat metastasized to the brain. The label information may include theorgan, such as brain, to which the cancer metastasized, the period oftime from initial diagnosis to metastasis diagnosis, such as a binaryyes or no to within the 12 months after sequencing, the staginginformation, tumor state such as TNM and so forth. RNA feature data maybe presented in a format representing raw reads from sequencing, rawreads ma RPKM, FPKM, TPM, or other methods as described above.

As an example, the artificial intelligence engine may be initiated for aTarget/Objective of Metastasis from Lung to Liver within 24 months. Amodel 146 b may be trained on labeled DNA feature data. The labeled DNAfeature data may be labeled with information about whether the patientfrom whom the DNA feature data was derived had a form of lung cancerthat metastasized to the liver. The label information may include theorgan, such as liver, to which the cancer metastasized, the period oftime from initial diagnosis to metastasis diagnosis, such as a binaryyes or no to within the 24 months after sequencing, the staginginformation, tumor state such as TNM and so forth. In an example, theDNA feature data may include features which are labeled on a structuralvariant by variant basis or features which are labeled on a gene by genebasis. One such example of a gene-by-gene basis includes the features ofTable 1, above.

In an example, the physician may order sequencing of a tumor specimenand blood sample from a laboratory where the ordering includesidentification of which tests the physician would like the laboratory toperform on the specimen and blood sample in addition to a report of thepatient's genomic profile. Tests may include MSI, TMB, CNV, Fusions,activated or deactivated pathways, or other alterations as describedabove with respect to alteration module 250. Tests may also includepredictions from an artificial intelligence engine, such as system 100for generating and modeling predictions of patient objectives, or morespecifically the metastasis site objective module 146 of ObjectiveModules 140. A physician may order sequencing with only genetic resultsand their implications reported or sequencing with genetic results andsubsequent alteration tests, including a prediction of patient'slikelihood for metastasis, reported together. In an example, a physicianmay order predictions without sequencing, such as when anotherlaboratory sequences the patient's tumor. The physician may submit thepatient's EMR or sequencing results for abstraction, and predictions maybe performed on the sequencing results only or the sequencing resultsand any abstracted patient features obtained from the EMR. Sequencingresults from another lab may include DNA, RNA, or Immunology relatedfeatures.

In an example, a tumor specimen may be difficult to obtain, such as fromthe brain or lung or when a patient is undergoing recurring sequencingand a physician may not desire to repeatedly biopsy the patient's tumor.In place of sampling the patient's tumor in some instances, a sample ofblood may be processed using a cell-free DNA, cfDNA, approach orsequencing of proteins using protein sequencers.

In an example, a patient may be identified as high risk for metastasisbased upon the patient's circumstances and a test for likelihood ofmetastasis may not be ordered. High risk circumstances may include NSCLCrecurrence, lack of actionable targeted mutations, expression of EGFR,EGFR T790M, ALK, ROS1, RET, BRAF, NTRK mutations which are typicallymore likely to metastasize, certain high risk combinations of cancertypes and patient features including colorectal cancer and highmicrosatellite instability, or based upon staging of the cancer.

As described above, models may be generated for any combination offeatures based upon the best performance to patients having arepresentative selection of features a model has been trained on. Eachpatient has a unique feature set based upon their interactions with themedical system and length of time in the medical system. While it isimpossible to exhaustively list every combination of features, patientstend to bin into a set of feature sets. As the medical industry advancesand more feature sets are curated for more patients, the models listedhere may be increased. Accordingly, a patient may be selected for amodel comprising features wherein the patient features include: raw RNAreads, normalized RNA reads, autoencoded RNA reads, RNA relatedfeatures, any RNA related features with any other RNA related features,DNA reads, normalized DNA reads, autoencoded DNA reads, DNA relatedfeatures, any DNA related features with any other DNA related features,any RNA related features with any DNA related features, RNA and DNAreads, RNA and DNA related features, RNA reads and imaging features, RNArelated features and imaging features, DNA reads and imaging features,DNA related features and imaging features, cfDNA reads, cfDNA relatedfeatures, cfDNA reads and imaging features, cfDNA related features andimaging features, cfDNA reads and clinical features, cfDNA relatedfeatures and clinical features, cfDNA reads and combined clinical andimaging features, cfDNA related features and combined clinical andimaging features, cfDNA related features and RNA related features, cfDNArelated features and DNA related features, combined RNA and DNA readsand imaging features, combined RNA and DNA related features and imagingfeatures, RNA reads and clinical features, RNA related features andclinical features, DNA reads and clinical features, DNA related featuresand clinical features, imaging features and clinical features, RNA readsand combined imaging and clinical features, RNA related features andcombined imaging and clinical features, DNA reads and combined imagingand clinical features, DNA related features and combined imaging andclinical features, combined RNA and DNA reads and combined imaging andclinical features, any combined RNA and DNA related features andcombined imaging and clinical features, any RNA related features withany proteomic features, and DNA related features with any proteomicfeatures, combined RNA and DNA related features with any proteomicfeatures, any DNA related features combined with imaging features andproteomic features, any DNA related features combined with clinicalfeatures and proteomic features, any RNA related features combined withimaging features and proteomic features, any RNA related featurescombined with clinical features and proteomic features, and any combinedRNA and DNA related features combined with combined clinical and imagingfeatures and proteomic features. While some combinations have beeninadvertently left out of the above listing of combinations of features,it should be appreciated that a full combinatorial listing of featuresfrom each of the above features to each other of the above features isattempted and desired. It should also be appreciated that a fullcombinatorial listing of the features from feature store 120 issimilarly disclosed as applicable models to the artificial intelligenceengine as disclosed herein.

It should be understood that RNA related features may include raw RNAreads, normalized RNA reads, and autoencoded RNA reads and that DNArelated features may include raw DNA reads, normalized DNA reads, andautoencoded DNA reads. Therefore combined RNA and DNA related featuresmay include any combination raw RNA reads to raw DNA reads, normalizedDNA reads, and autoencoded DNA reads, normalized RNA reads to raw DNAreads, normalized DNA reads, and autoencoded DNA reads, autoencoded RNAreads to raw DNA reads, normalized DNA reads, and autoencoded DNA readsand vice versa.

The methods and systems described above may be utilized in combinationwith or as part of a digital and laboratory health care platform that isgenerally targeted to medical care and research, and in particular,generating a molecular report as part of a targeted medical careprecision medicine treatment or research. It should be understood thatmany uses of the methods and systems described above, in combinationwith such a platform, are possible. An example of such a platform isdescribed in U.S. patent application Ser. No. 16/657,804, titled “DataBased Cancer Research and Treatment Systems and Methods” (hereinafter“the '804 application”), which is incorporated herein by reference inits entirety for all purposes. In some aspects, a physician or otherindividual may utilize an artificial intelligence engine, such as thesystem 100 for generating and modeling predictions of patientobjectives, in connection with one or more expert treatment systemdatabases shown in FIG. 1 of the '804 application. The artificialintelligence engine of system 100 may operate on one or moremicro-services operating as part of systems, services, applications, andintegration resources database, and the methods described herein may beexecuted as one or more system orchestration modules/resources,operational applications, or analytical applications. At least some ofthe methods (e.g., microservices) can be implemented as computerreadable instructions that can be executed by one or more computationaldevices, such as the artificial intelligence engine of system 100. Forexample, an implementation of one or more embodiments of the methods andsystems as described above may include microservices included in adigital and laboratory health care platform that can generatepredictions of a patient's likelihood to metastasis to an organ within atime period based upon the patient's available features and sequencingresults.

In some embodiments, a system may include a single microservice forexecuting and delivering the predictions or may include a plurality ofmicroservices, each microservice having a particular role which togetherimplement one or more of the embodiments above. In an example, a firstmicroservice may include extracting patient information from one or morepatients, identifying one or more interactions for each of the one ormore patients based at least in part on the received patientinformation; generating, for one or more targets at each one or moreinteractions, one or more timeline metrics identifying whether each ofthe one or more targets occurs within a time period of an occurrence ofthe interaction; identifying, for each timeline metric of the one ormore timeline metrics, whether a patient will be associated with one ormore status characteristics within the time period; training a targetprediction model for each of the one or more targets based at least inpart on the one or more status characteristics; and associatingpredictions for each patient from the target prediction model for eachof the one or more targets with a respective one or more timelinemetrics of the one or more timeline metrics. A second microservice mayinclude listening for an order to generate a prediction using theartificial intelligence engine of system 100 for a new patient using thetrained model. Similarly, the second microservice may include providingthe received information to the trained prediction model for theidentified target/objective and generating a prediction so that theartificial intelligence engine of system 100 may provide the predictionin response to the order according to an embodiment, above.

The artificial intelligence engine of system 100 may be utilized as asource for automated data generation of the kind identified in FIG. 59of the '804 application. For example, the artificial intelligence engineof system 100 may interact with an order intake server to receive anorder for a test, such as a test that provides predictions with respectto a patient. Where embodiments above are executed in one or moremicro-services with or as part of a digital and laboratory health careplatform, one or more of such micro-services may be part of an ordermanagement system that orchestrates the sequence of events as needed atthe appropriate time and in the appropriate order necessary toinstantiate embodiments above.

For example, continuing with the above first and second microservices,an order management system may notify the first microservice that anorder for a test has been received and is ready for processing. Thefirst microservice may include executing and notifying the ordermanagement system once the delivery of any patient information for thesecond microservice is ready, including one or more interactions, one ormore timeline metrics, and a target/objective pair. Furthermore, theorder management system may identify that execution parameters(prerequisites) for the second microservice are satisfied, includingthat the first microservice has completed, and notify the secondmicroservice that it may continue processing the order to provide theprediction from the artificial intelligence engine of system 100according to an embodiment, above. While two microservices are utilizedfor illustrative purposes, patient information extraction, interactionidentification, status characteristic identification, model training,and patient predictions may be split up between any number ofmicroservices in accordance with performing embodiments herein.

The digital and laboratory health care platform further includes one ormore insight engines shown in FIG. 272 of the '804 application.Exemplary insight engines may include a tumor of unknown origin engine,a human leukocyte antigen (HLA) loss of homozygosity (LOH) engine, atumor mutational burden (TMB) engine, a PD-L1 status engine, ahomologous recombination deficiency (HRD) engine, a cellular pathwayactivation report engine, an immune infiltration engine, amicrosatellite instability engine, a pathogen infection status engine,and so forth as described with respect to FIGS. 189, 199-200, and266-270 of the '804 application. In an aspect, a model may be trained onand subsequently receive as an input for predictions, features includingdiagnosis of the patient as to an insight engine such as HLA LOH, TMB,PD-L1, HRD, active pathway, or other insight status. The artificialintelligence engine of system 100 may identify a patient having featuresfrom an insight engine and select an appropriate model and feature setto utilize the features in a prediction.

When the digital and laboratory health care platform further includes amolecular report generation engine, the methods and systems describedabove may be utilized to create a summary report of a patient's geneticprofile and the results of one or more insight engines for presentationto a physician. For instance, the report may provide to the physicianinformation about the extent to which the specimen that was sequencedcontained tumor or normal tissue from a first organ, a second organ, athird organ, and so forth. For example, the report may provide a geneticprofile for each of the tissue types, tumors, or organs in the specimen.The genetic profile may represent genetic sequences present in thetissue type, tumor, or organ and may include variants, expressionlevels, information about gene products, or other information that couldbe derived from genetic analysis of a tissue, tumor, or organ via agenetic analyzer. The report may further include therapies and/orclinical trials matched based on a portion or all of the genetic profileor insight engine findings and summaries shown in FIGS. 271 and 302 ofthe '804 application.

It should be understood that the examples given above are illustrativeand do not limit the uses of the systems and methods described herein incombination with a digital and laboratory health care platform.

Predicting Metastasis of Cancer in a Subject.

Example System Embodiments

Now that an overview of some aspects of the present disclosure have beenprovided, details of an exemplary system are described in conjunctionwith FIG. 15. FIG. 15 is a block diagram illustrating a system 1500 inaccordance with some implementations.

The system 1500 in some implementations includes one or more processingunits CPU(s) 1502 (also referred to as processors), one or more networkinterfaces 104, a user interface 106 including (optionally) a display1508 and an input system 1510, a non-persistent memory 1511, apersistent memory 1512, and one or more communication buses 1514 forinterconnecting these components. The one or more communication buses1514 optionally include circuitry (sometimes called a chipset) thatinterconnects and controls communications between system components. Thenon-persistent memory 1511 typically includes high-speed random accessmemory, such as DRAM, SRAM, DDR RAM, ROM, EEPROM, flash memory, whereasthe persistent memory 1512 typically includes CD-ROM, digital versatiledisks (DVD) or other optical storage, magnetic cassettes, magnetic tape,magnetic disk storage or other magnetic storage devices, magnetic diskstorage devices, optical disk storage devices, flash memory devices, orother non-volatile solid state storage devices. The persistent memory1512 optionally includes one or more storage devices remotely locatedfrom the CPU(s) 1502. The persistent memory 1512, and the non-volatilememory device(s) within the non-persistent memory 1512, comprisenon-transitory computer readable storage medium.

In some implementations, as illustrated in FIG. 15, the non-persistentmemory 1511 or alternatively the non-transitory computer readablestorage medium stores the following programs, modules and datastructures, or a subset thereof, sometimes in conjunction with thepersistent memory 1512:

-   -   an optional operating system 1516, which includes procedures for        handling various basic system services and for performing        hardware dependent tasks;    -   an optional network communication module (or instructions) 1518        for connecting the system 1500 with other devices and/or a        communication network 1504;    -   an optional classifier training module 1520 for training one or        more models (e.g., predictive and/or classification models) to        provide one or more indications of whether the cancer will        metastasize in the subject;    -   a data structure 1522 comprising a plurality of data elements        for a cancer of a subject 1524 (e.g., optionally, a plurality of        subjects 1524-1, . . . , 1524-K), the data structure 1522        comprising:        -   a sequence features data construct 1526 (e.g., 1526-1)            comprising a first set of sequence features 1528 (e.g.,            1528-1-1, . . . , 1528-1-L) (e.g., relative abundance values            for the expression of a plurality of genes (e.g., at least            30 genes) in a biopsy of the cancer obtained from the            subject);        -   optionally, a personal characteristics data construct 1530            (e.g., 1530-1) comprising one or more personal            characteristics about the subject (e.g., age, gender, and/or            race); and        -   optionally, a clinical features data construct 1532 (e.g.,            1532-1) comprising one or more clinical features related to            the diagnosis or treatment of the cancer in the subject            and/or one or more temporal elements associated with the one            or more clinical features;    -   a classification module 1536 comprising one or more models 1538        (e.g., optionally, a set of models) that are trained to provide        one or more indications of whether the cancer will metastasize        in the subject; and    -   an output module 1540 comprising one or more indications 1542 of        whether the cancer will metastasize in the subject (e.g.,        optionally, for each respective tissue in a plurality of        tissues, a respective set of indications for each respective        time horizon in a plurality of time horizons).

In various implementations, one or more of the above identified elementsare stored in one or more of the previously mentioned memory devices,and correspond to a set of instructions for performing a functiondescribed above. The above identified modules, data, or programs (e.g.,sets of instructions) need not be implemented as separate softwareprograms, procedures, datasets, or modules, and thus various subsets ofthese modules and data may be combined or otherwise re-arranged invarious implementations. In some implementations, the non-persistentmemory 1511 optionally stores a subset of the modules and datastructures identified above. Furthermore, in some embodiments, thememory stores additional modules and data structures not describedabove. In some embodiments, one or more of the above identified elementsis stored in a computer system, other than that of system 1500, that isaddressable by system 1500 so that system 1500 may retrieve all or aportion of such data when needed.

Although FIG. 15 depicts a “system 1500,” the figure is intended more asa functional description of the various features that may be present incomputer systems than as a structural schematic of the implementationsdescribed herein. In practice, and as recognized by those of ordinaryskill in the art, items shown separately could be combined and someitems could be separated. Moreover, although FIG. 15 depicts certaindata and modules in non-persistent memory 1511, some or all of thesedata and modules instead may be stored in persistent memory 1512 or inmore than one memory. For example, in some embodiments, at least datastructure 1522 is stored in a remote storage device which can be a partof a cloud-based infrastructure. In some embodiments, at least datastructure 1522 is stored on a cloud-based infrastructure. In someembodiments, data structure 1522, the classifier training module 1520,the classification module 1536, and/or the output module 1540 can alsobe stored in the remote storage device(s). In some embodiments, any ofthe features of the system 1500 can be used in conjunction with any ofthe features of any one or more of system 100, system 200, system 300,system 400, system 500, and/or system 1400, as depicted in FIGS. 1-5 and14, and/or any combinations thereof as will be apparent to one skilledin the art. For instance, data structure 1522 can comprise any of theexample features listed in systems 400 and 500 or can further beassociated with any one or more of feature collection 205, featuremodule 110, feature store 120, feature selector 200, and featuregenerator 300.

Classification Methods

While a system in accordance with the present disclosure has beendisclosed with reference to FIG. 15 (e.g., alone or in combination withany one or more of FIGS. 1-5 and 14), methods in accordance with thepresent disclosure are now detailed with reference to FIGS. 16 and 17.

Referring to FIG. 16, the present disclosure provides a method 1602 forpredicting metastasis of a cancer in a subject. In particular, in someembodiments, the systems and methods described herein utilize at leastrelative abundance values determined from RNA molecules in a sample fromthe subject to determine at least a first indication (e.g., a binaryindication and/or a probability) of whether the cancer will metastasizein the subject. In some embodiments, the systems and methods describedherein are used to determine an indication (e.g., a binary indicationand/or a probability) of whether the cancer will metastasize to arespective tissue in the subject, for each tissue in a plurality oftissues. In some embodiments, the systems and methods described hereinare used to determine an indication of whether the cancer willmetastasize within a respective time horizon (e.g., a period of timefollowing a temporal element such as a clinical event), for each timehorizon in a plurality of time horizons. In some embodiments, thesystems and methods disclosed herein are used to determine an indication(e.g., a binary indication and/or a probability) of whether the cancerwill metastasize to any given tissue within any given time horizon, foreach respective tissue in a plurality of tissues and each time horizonin a plurality of time horizons. In some such embodiments, any possiblecombination of tissue and time horizon that will be apparent to oneskilled in the art is contemplated for the determination and/orprovision of metastasis predictions.

Benefit.

In some embodiments, the systems and methods described herein providepredictions of the likelihood, location, and/or projected timeline ofcancer metastasis that are used to improve patient diagnosis, clinicalreporting, monitoring of cancer progression, assessments of treatmentefficacy, recommendations of personalized therapies, and/oradministration of personalized therapies (e.g., targeted cancertherapies). For example, conventional methods of cancer therapy arebased upon generalized standard of care guidelines that are applicableacross a wide variety of patients and clinical histories (e.g., aone-size-fits-all care regimen). By contrast, predictions of metastasisimprove upon conventional methods of cancer therapy by consideringpatient-specific molecular, demographics, and/or clinical features todetermine a personalized mode of treatment that is designed to treat notonly the current stage or type of cancer, but also to anticipate andtreat, prevent, and/or ameliorate metastatic cancers.

As such, predictions of the likely occurrence, time, and/or location ofcancer metastasis allows patients to access (e.g., via a physician ormedical provider) more expedient, affordable care as part of thebenefits of precision medicine. Furthermore, as described above,applications of predictions of cancer metastasis based on such inputs asgenetic and molecular sequencing, personal characteristics and/orclinical data can be used to identify therapies which are expected toperform well for a patient having characteristics similar to thereported patient, clinical trials which may accept the patient, or otherpersonalized analysis results which may influence the physician'sdecisions. Notably, a prediction of cancer metastasis can be used as thebasis for prescribing a treatment which is considered aggressive for thetreatment and prevention of metastasis. Alternatively, a prediction thatmetastasis is unlikely within an extended target window (e.g., the next60 months) can be used as the basis for suggesting a less aggressivetreatment to the patient which may be more cost effective and/or causeless severe, or avoid altogether, adverse effects associated with moreaggressive treatment regimen.

The benefits of predicting the likelihood of cancer metastasis are notlimited to treatments, but can be used to schedule the frequency ofmonitoring for the patient, such as follow-up visits, additionalscanning, screening, imaging, blood tests, or subsequent geneticsequencing. For example, a patient with a high prediction of metastasismay benefit from accelerated screening to detect changes at regularintervals, allowing early intervention before the onset of severe,advanced, and/or noticeable side effects. Additionally, metastasispredictions can be used to determine enrollment in drug trials and/orclinical trials (e.g., by a pharmaceutical company) based on theintersection between inclusion and exclusion criteria and theprobability that the patient will experience a predicted outcome, and toprovide valuable data, upon analysis, as to the underlying genetic,demographic, and/or clinical basis for positive or negative responses toexperimental treatment.

In some embodiments, the systems and methods described herein support aclinical tool for assisting with clinical care, such as a graphical userinterface (GUI) and/or tool for determination, reporting, andvisualization of one or more indications of cancer metastasis. In someembodiments, the use of a cancer metastasis prediction tool informsusers (e.g., clinicians, researchers, and/or patients) as to likelyevents throughout cancer progression and further provides guidance fordecision-making with respect to standard of care therapyrecommendations, clinical trial enrollment, and reimbursement fortherapies.

It should be noted that details of other processes described herein (asin the above sections, “Generating and Modeling Predictions of PatientObjectives”) are also applicable in an analogous manner to the methodsdescribed below with reference to FIGS. 16 and 17. For example, detailsrelating to features (e.g., types of features, feature selectionmethodologies, RNA transcripts, genes, and/or dimensionality reduction),model selection, model training (e.g., artificial intelligence trainingpipeline), reporting (e.g., graphical user interfaces), etc., describedbelow with reference to method 1602 optionally have one or more of thecharacteristics of the features, model selection, model training,reporting, etc., described herein with reference to the above sections(e.g., “Generating and Modeling Predictions of Patient Objectives”). Forbrevity, these details are not repeated here.

Cancer Conditions.

In some embodiments, the subject is a human. In some embodiments, thesubject is a patient diagnosed with a cancer condition (e.g., a presenceor absence of cancer, a stage of cancer, a cancer type, a cancersubtype, a tissue of origin, a cancer grade, and/or a histopathologicalgrade). In some embodiments, the subject has a cancer having one or moreother cancer characteristics (e.g., a homologous recombinationdeficiency status, a microsatellite stability status, a mutationalburden, a genomic alteration (e.g., SNPs/indels, fusions, copy numbervariations, amplifications, deletions, and/or variant allelic ratios),and/or a genomic marker status (e.g., associated with an actionabletherapy)), which are further utilized to determine at least the firstindication of whether the cancer will metastasize in the subject.Methods for determining cancer characteristics are known in the art, asdescribed in U.S. patent application Ser. No. 15/930,234, filed May 12,2020, entitled “Systems and Methods for Multi-Label CancerClassification,” which is hereby incorporated by reference in itsentirety.

In some embodiments, the subject's cancer type has been determined. Insome such embodiments, a classifier trained specifically to predictmetastasis for that type of cancer is used. In other such embodiments, amulti-label classifier that uses cancer type as an input feature isused. In yet other such embodiments, a pan-cancer classifier that doesnot consider the type of cancer is used. In some embodiments, thesubject's cancer has not been identified, e.g., the subject has beendiagnosed with a cancer of unknown origin. Accordingly, in some suchembodiments, a pan-cancer classifier that does not consider the type ofcancer is used.

Generally, the methods and systems described herein are useful forpredicting metastasis for any type of cancer. In some embodiments, aclassifier used in the methods and systems described herein is trainedto predict metastasis for a single type of cancer. In other embodiments,the classifier is a multi-label classifier, that is trained to predictmetastasis for multiple types of cancer, e.g., which may or may not usethe subject's cancer type as an input. In yet other embodiments, theclassifier is a pan-cancer classifier that does not consider thesubject's cancer type when predicting metastasis for a subject.

In some embodiments, the cancer is colorectal cancer, non-small celllung cancer (NSCLC), breast cancer, or ovarian cancer.

In some embodiments, the cancer is selected from Acute LymphoblasticLeukemia (ALL), Acute Myeloid Leukemia (AML), Adolescents, Cancer in,Adrenocortical Carcinoma, AIDS-Related Cancers, Kaposi Sarcoma (SoftTissue Sarcoma), AIDS-Related Lymphoma (Lymphoma), Primary CNS Lymphoma(Lymphoma), Anal Cancer, Appendix Cancer, Astrocytomas, Childhood (BrainCancer), Atypical Teratoid/Rhabdoid Tumor, Childhood, Central NervousSystem (Brain Cancer), Basal Cell Carcinoma of the Skin, Bile DuctCancer, Bladder Cancer, Bone Cancer (includes Ewing Sarcoma andOsteosarcoma and Malignant Fibrous Histiocytoma), Brain Tumors, BreastCancer, Bronchial Tumors (Lung Cancer), Burkitt Lymphoma, CarcinoidTumor (Gastrointestinal), Carcinoma of Unknown Primary, Cardiac (Heart)Tumors, Childhood, Central Nervous System, Atypical Teratoid/RhabdoidTumor, Childhood (Brain Cancer), Medulloblastoma and Other CNS EmbryonalTumors, Childhood (Brain Cancer), Germ Cell Tumor, Childhood (BrainCancer), Primary CNS Lymphoma, Cervical Cancer, Childhood Cancers,Cancers of Childhood, Unusual, Cholangiocarcinoma, Chordoma, Childhood(Bone Cancer), Chronic Lymphocytic Leukemia (CLL), Chronic MyelogenousLeukemia (CML), Chronic Myeloproliferative Neoplasms, Colorectal Cancer,Craniopharyngioma, Childhood (Brain Cancer), Cutaneous T-Cell Lymphoma,Ductal Carcinoma In Situ (DCIS), Childhood (Brain Cancer), EndometrialCancer (Uterine Cancer), Ependymoma, Childhood (Brain Cancer),Esophageal Cancer, Esthesioneuroblastoma (Head and Neck Cancer), EwingSarcoma (Bone Cancer), Extracranial Germ Cell Tumor, Childhood,Extragonadal Germ Cell Tumor, Eye Cancer, Intraocular Melanoma,Retinoblastoma, Fallopian Tube Cancer, Fibrous Histiocytoma of Bone,Malignant, and Osteosarcoma, Gallbladder Cancer, Gastric (Stomach)Cancer, Gastrointestinal Carcinoid Tumor, Gastrointestinal StromalTumors (GIST) (Soft Tissue Sarcoma), Germ Cell Tumors, Childhood CentralNervous System Germ Cell Tumors (Brain Cancer), Childhood ExtracranialGerm Cell Tumors, Extragonadal Germ Cell Tumors, Ovarian Germ CellTumors, Testicular Cancer, Gestational Trophoblastic Disease, Hairy CellLeukemia, Head and Neck Cancer, Heart Tumors, Childhood, Hepatocellular(Liver) Cancer, Histiocytosis, Langerhans Cell, Hodgkin Lymphoma,Hypopharyngeal Cancer (Head and Neck Cancer), Intraocular Melanoma,Islet Cell Tumors, Pancreatic Neuroendocrine Tumors, Kaposi Sarcoma(Soft Tissue Sarcoma), Kidney (Renal Cell) Cancer, Langerhans CellHistiocytosis, Laryngeal Cancer (Head and Neck Cancer), Leukemia, Lipand Oral Cavity Cancer (Head and Neck Cancer), Liver Cancer, Lung Cancer(Non-Small Cell, Small Cell, Pleuropulmonary Blastoma, andTracheobronchial Tumor), Lymphoma, Male Breast Cancer, Malignant FibrousHistiocytoma of Bone and Osteosarcoma, Melanoma, Melanoma, Intraocular(Eye), Merkel Cell Carcinoma (Skin Cancer), Mesothelioma, Malignant,Metastatic Cancer, Metastatic Squamous Neck Cancer with Occult Primary(Head and Neck Cancer), Midline Tract Carcinoma With NUT Gene Changes,Mouth Cancer (Head and Neck Cancer), Multiple Endocrine NeoplasiaSyndromes, Multiple Myeloma/Plasma Cell Neoplasms, Mycosis Fungoides(Lymphoma), Myelodysplastic Syndromes,Myelodysplastic/Myeloproliferative Neoplasms, Myelogenous Leukemia,Chronic (CML), Myeloid Leukemia, Acute (AML), MyeloproliferativeNeoplasms, Chronic, Nasal Cavity and Paranasal Sinus Cancer (Head andNeck Cancer), Nasopharyngeal Cancer (Head and Neck Cancer),Neuroblastoma, Non-Hodgkin Lymphoma, Non-Small Cell Lung Cancer, OralCancer, Lip and Oral Cavity Cancer and Oropharyngeal Cancer (Head andNeck Cancer), Osteosarcoma and Malignant Fibrous Histiocytoma of Bone,Ovarian Cancer, Pancreatic Cancer, Pancreatic Neuroendocrine Tumors(Islet Cell Tumors), Papillomatosis (Childhood Laryngeal),Paraganglioma, Paranasal Sinus and Nasal Cavity Cancer (Head and NeckCancer), Parathyroid Cancer, Penile Cancer, Pharyngeal Cancer (Head andNeck Cancer), Pheochromocytoma, Pituitary Tumor, Plasma CellNeoplasm/Multiple Myeloma, Pleuropulmonary Blastoma (Lung Cancer),Pregnancy and Breast Cancer, Primary Central Nervous System (CNS)Lymphoma, Primary Peritoneal Cancer, Prostate Cancer, Rectal Cancer,Recurrent Cancer, Renal Cell (Kidney) Cancer, Retinoblastoma,Rhabdomyosarcoma, Childhood (Soft Tissue Sarcoma), Salivary Gland Cancer(Head and Neck Cancer), Childhood Rhabdomyosarcoma (Soft TissueSarcoma), Childhood Vascular Tumors (Soft Tissue Sarcoma), Ewing Sarcoma(Bone Cancer), Kaposi Sarcoma (Soft Tissue Sarcoma), Osteosarcoma (BoneCancer), Soft Tissue Sarcoma, Uterine Sarcoma, Sézary Syndrome(Lymphoma), Skin Cancer, Small Cell Lung Cancer, Small Intestine Cancer,Soft Tissue Sarcoma, Squamous Cell Carcinoma of the Skin, Squamous NeckCancer with Occult Primary, Metastatic (Head and Neck Cancer), Stomach(Gastric) Cancer, T-Cell Lymphoma, Lymphoma (Mycosis Fungoides andSèzary Syndrome), Testicular Cancer, Throat Cancer (Head and NeckCancer), Nasopharyngeal Cancer, Oropharyngeal Cancer, HypopharyngealCancer, Thymoma and Thymic Carcinoma, Thyroid Cancer, TracheobronchialTumors (Lung Cancer), Transitional Cell Cancer of the Renal Pelvis andUreter (Kidney (Renal Cell) Cancer), Ureter and Renal Pelvis,Transitional Cell Cancer (Kidney (Renal Cell) Cancer, Urethral Cancer,Uterine Cancer, Endometrial, Uterine Sarcoma, Vaginal Cancer, VascularTumors (Soft Tissue Sarcoma), and/or Vulvar Cancer.

In some embodiments, the cancer is selected from brain non-glioma(ependymoma, hemangioblastoma, medulloblastoma, meningioma), breast(breast ductal, breast lobular), colon, endometrial (endometrial,endometrial serous, endometrial stromal sarcoma), gastroesophageal(esophageal adenocarcinoma, gastric), gastrointestinal stromal tumor,glioma (Glioma, oligodendroglioma), head and neck adenocarcinoma,hematological (acute lymphoblastic leukemia, acute myeloid leukemia, bcell lymphoma, chronic lymphocytic leukemia, chronic myeloid leukemia,rosai dorfman, T-cell lymphoma), hepatobiliary (cholangiocarcinoma,gallbladder, liver), lung adenocarcinoma, melanoma, mesothelioma,neuroendocrine (gastrointestinal neuroendocrine, high gradeneuroendocrine lung, low grade neuroendocrine lung, pancreaticneuroendocrine, skin neuroendocrine), ovarian (ovarian clear cell,ovarian granulosa, ovarian serous), pancreas, prostate, renal (renalchromophobe, renal clear cell, renal papillary), sarcoma(chondrosarcoma, chordoma, ewing sarcoma, fibrous sarcoma,leiomyosarcoma, liposarcoma, osteosarcoma, rhabdomyosarcoma, synovialsarcoma, vascular sarcoma), squamous (cervical, esophageal squamous,head and neck squamous, lung squamous, skin squamous/basal), thymic,thyroid, and/or urothelial cancers.

In some embodiments, the cancer is any one or more entries of theICD-10-CM, or the International Classification of Disease. The ICDprovides a method of classifying diseases, injuries, and causes ofdeath. The World Health Organization (WHO) publishes the ICDs tostandardize the methods of recording and tracking instances of diagnoseddisease, including cancer. For example, in some embodiments, the canceris selected from the classifications from any chapter of the ICD orcancers from Chapter 2, C and D codes. C codes include Neoplasm of Lip,Oral Cavity and Pharynx (C00-C14), Neoplasm of Digestive Organs(C15-C26), Neoplasm of Respiratory System and Intrathoracic Organs(C30-C39), Neoplasm of Mesothelial and Soft Tissue (C45), Neoplasm ofBones, Joints and Articular Cartilage (C40-C41), Neoplasm of Skin(Melanoma, Merkel Cell, and Other Skin Histologies) (C43, C44, C4a),Kaposi Sarcoma (46), Neoplasm of Peripheral Nerves and Autonomic nervoussystem, Retroperitoneum, Peritoneum, and Soft Tissues (C47, C48, C49),Neoplasm of Breast and Female Genital Organs (C50-C58), Neoplasm of MaleGenital Organs (C60-C63), Neoplasm of Urinary Tract (C64-C68), Neoplasmsof Eye, Brain and Other Parts of the Central Nervous System (C69-C72),Neoplasm of Thyroid, Other Endocrine Glands, and Ill-defined Sites(C73-C76), Malignant Neuroendocrine Tumors (C7a._), SecondaryNeuroendocrine Tumors (C7B), Neoplasm of other and ill-defined sites(C76-80), Secondary and unspecified malignant neoplasm of lymph nodes(C77), Secondary Cancers of respiratory and digestive organs, other andunspecified sites (C78-80), Malignant Neoplasm without specification ofsite (C80), Malignant neoplasms of lymphoid, and/or hematopoietic andrelated tissue (C81-C96).

In some embodiments, the cancer is any broadly construed categorizationto a cohort class. Exemplary cohort classes include, but are not limitedto, Blood Cancer, Bone Cancer, Brain Cancer, Bladder Cancer, BreastCancer, Colon and Rectal Cancer, Endometrial Cancer, Kidney Cancer,Leukemia, Liver Cancer, Lung Cancer, Melanoma, Non-Hodgkin Lymphoma,Pancreatic Cancer, Prostate Cancer, Thyroid Cancer, and/or othertissue-based or organ-based classifications.

In some embodiments, the cancer is a site of biopsy for a biopsyspecimen (e.g., a sample from a subject) such as one or more ICD-03codes, including but not limited to lip, base of tongue, tongue, gum,floor of mouth, other mouth, salivary gland, oropharynx, nasopharynx,posterior wall of nasopharynx, hypopharynx, pharynx, esophagus, stomach,small intestine, large intestine, appendix, rectum, anal canal and/oranus, liver, intrahepatic bile ducts, gallbladder and/or extrahepaticbile ducts, pancreas, unspecified digestive organs, nasal cavity(including nasal cartilage), middle ear, sinuses, accessory sinus, nose,larynx, trachea, lung and/or bronchus, thymus, heart, mediastinum,pleura, respiratory, bones and/or joints, bones of skull and face,mandible, blood, bone marrow, hematopoietic system, spleen,reticulo-endothelial, skin, peripheral nerves, retroperitoneum and/orperitoneum, connective and/or soft tissue, breast, vagina and/or labia,vulva, cervix uteri, corpus uteri, uterus, ovary, fallopian tube, otherfemale genital, placenta, penis, prostate gland, testis, epididymis,spermatic cord, male genital, scrotum, kidney, renal pelvis, ureter,urinary bladder, other urinary organs, orbit and/or lacrimal gland,retina, eyeball, eye, nose, meninges (e.g., cerebral and spinal), brain,cranial nerves, spinal cord, ventricle, cerebellum, other nervoussystem, thyroid gland, adrenal glands, parathyroid gland, pituitarygland, craniopharyngeal duct, pineal gland, other endocrine glands,and/or lymph nodes.

In some embodiments, the cancer is one of a plurality of tumor and/ortissue types having common cell lineages. In some embodiments, thecancer is one of a plurality of metastasis sites and/or a metastasissite of origin (e.g., a liver metastasis of pancreatic origin, uppergastrointestinal origin, or cholangio origin; a breast metastasis ofsalivary gland origin, squamous origin, or ductile origin; a brainmetastasis of glioblastoma, oligodendroglioma, astrocytoma, ormedulloblastoma; a lung metastasis of NSCLC adenocarcinoma or squamous,etc.).

Data Elements.

Generally, the methods and systems described herein use severaldifferent types of characteristics about the subject's cancer and/or thesubject themselves, to predict whether the subject's cancer willmetastasize. In some embodiments, these characteristics include at leastone type of sequence feature determined from sequencing of nucleic acidsfrom the subject's cancer (e.g., DNA and/or RNA from the subject'scancer). As described above, different types of biological samples fromthe subject can provide such nucleic acids, including biopsies of thecancerous tissue and blood samples which contain circulating tumor DNA(ctDNA). In some embodiments, these characteristics include one or morepersonal characteristics about the subject (e.g., age, gender, race, andothers described herein). In some embodiments, these characteristicsinclude one or more clinical features related to the diagnosis and/ortreatment of the cancer (e.g., including temporal and/or non-temporalclinical features).

Referring to Block 1604, in some embodiments, the method includesobtaining, in electronic format, a plurality of data elements for thesubject's cancer. The plurality of data elements includes a first set ofsequence features (1606) e.g., relative abundance values for theexpression of a plurality of genes (e.g., at least 30 genes) in a biopsyof the cancer obtained from the subject. For example, in someembodiments, the plurality of data elements is stored in a datastructure 1522 in a system 1500, and the first set of sequence features(1606) comprises a plurality of sequence features 1528 stored in asequence features construct 1526. Additionally, as illustrated in FIG.17 and discussed in further detail below, in some embodiments, the firstset of sequence features includes nucleic acid-based features (e.g., RNAexpression, tumor purity, germline genomics, and/or cancer genomics).

In some embodiments, the plurality of data elements includes one or morepersonal characteristics (1608) about the subject (e.g., age, gender,race, habits, and/or biological statuses; see also FIG. 17). In someembodiments, the one or more personal characteristics is stored in anoptional personal characteristics construct 1530 in a data structure1522.

In some embodiments, the plurality of data elements includes one or moreclinical features (1610) related to the diagnosis or treatment of thecancer in the subject (e.g., a stage of the cancer, a histopathologicalgrade of the cancer, a diagnosis for the cancer, a therapy administeredto the subject, a symptom associated with cancer or metastasis thereof,a comorbidity with the cancer, and/or one or more temporal elementsassociated therewith). In some embodiments, the plurality of dataelements includes one or more temporal features (1611) related to thediagnosis or treatment of the cancer in the subject (e.g., a firsttemporal element indicating a duration of time since a diagnosis for thecancer, a second temporal element indicating a duration of time since anadministration of a therapy to the subject, a third temporal elementindicating a duration of time since an experience of a symptomassociated with cancer or metastasis thereof, and/or a fourth temporalelement indicating a duration of time since an experience of acomorbidity with the cancer). For example, as illustrated in FIG. 17, insome embodiments, the one or more clinical features includescharacteristics of cancer (e.g., primary cancer type, cancer stage,therapy, symptoms, and/or comorbidity) and one or more temporal elementsassociated therewith (e.g., time since diagnosis, time since therapy,time since symptoms, and/or time since comorbidity). In someembodiments, the one or more clinical features including characteristicsof cancer and temporal elements are stored in an optional clinicalfeatures construct 1532 in a data structure 1522.

Sequence features 1606. In some embodiments, the plurality of dataelements includes a first set of sequence features (1606) (e.g., storedin a sequence features construct 1526) determined from sequencing ofnucleic acids from the subject's cancer.

In some embodiments, the nucleic acids are from a biopsy of the cancerobtained from the subject. Methods for biopsy collection are known inthe art, including, but not limited to, macrodissected formalin fixedparaffin embedded (FFPE) tissue sections, surgical biopsy, skin biopsy,punch biopsy, prostate biopsy, bone biopsy, bone marrow biopsy, needlebiopsy, CT-guided biopsy, ultrasound-guided biopsy, fine needleaspiration, aspiration biopsy, fresh tissue and/or blood samples. Insome embodiments, the biopsy is a solid sample of a cancerous tissuefrom the subject. In some embodiments, the biopsy is a liquid sample ofa blood cancer. In some embodiments, the biopsy is a liquid samplecontaining ctDNA from a solid cancerous tissue of the subject. In someembodiments, sequence features are determined using both a cancerousbiopsy and a tumor-matched sample of non-cancerous tissue from thesubject.

In some embodiments, the first set of sequence features includes atleast 25, at least 30, at least 40, at least 50, at least 75, at least100, at least 150, at least 200, at least 250, at least 300, at least400, at least 500, at least 600, at least 700, at least 800, at least900, at least 1000, or more sequence features. In some embodiments, thefirst set of sequence features for the subject includes at least 100, atleast 200, at least 300, at least 400, at least 500, at least 600, atleast 700, at least 800, at least 900, at least 1000, at least 1500, atleast 2000, at least 2500, at least 3000, at least 4000, at least 5000,at least 7500, at least 10,000, at least 15,000, at least 20,000, atleast 25,000, at least 30,000 or more sequence features.

In some embodiments, the first set of sequence features includes no morethan 50, no more than 75, no more than 100, no more than 150, no morethan 200, no more than 250, no more than 300, no more than 400, no morethan 500, no more than 600, no more than 700, no more than 800, no morethan 900, or no more than 1000 sequence features. In some embodiments,the first set of sequence features for the subject includes no more than100, no more than 200, no more than 300, no more than 400, no more than500, no more than 600, no more than 700, no more than 800, no more than900, no more than 1000, no more than 1500, no more than 2000, no morethan 2500, no more than 3000, no more than 4000, no more than 5000, nomore than 7500, no more than 10,000, no more than 15,000, no more than20,000, no more than 25,000, or no more than 30,000 sequence features.

In some embodiments, the first set of sequence features includes from 25to 5000 sequence features. In some embodiments, the first set ofsequence features includes from 50 to 1000 sequence features. In someembodiments, the first set of sequence features includes from 75 to 500sequence features. In some embodiments the first set of sequencefeatures falls within another range starting no lower than 25 sequencefeatures and ending no higher than 30,000 sequence features.

In some embodiments, the first set of sequence features are determinedfrom a first plurality of sequence reads obtained from a sequencingreaction of nucleic acids from a biopsy of a cancerous tissue from thesubject. In some embodiments, the first plurality of sequence reads isobtained by any suitable sequencing method, including but not limited tonext-generation sequencing, whole genome sequencing, targeted panelsequencing, RNA sequencing, mRNA sequencing, whole exome sequencing,microarrays, and/or quantitative reverse transcription polymerase chainreaction. For example, methods of next-generation sequencing for use inaccordance with the methods described herein are disclosed in Shendure,2008, Nat. Biotechnology 26:1135-1145 and Fullwood et al., 2009, GenomeRes. 19:521-532, each of which is hereby incorporated herein byreference in its entirety. Next generation sequencing methods well knownin the art include synthesis technology (Illumina), pyrosequencing (454Life Sciences), ion semiconductor technology (Ion Torrent sequencing),single-molecule real-time sequencing (Pacific Biosciences), sequencingby ligation (SOLiD sequencing), nanopore sequencing (Oxford NanoporeTechnologies), or paired-end sequencing. In some embodiments, massivelyparallel sequencing is performed using sequencing-by-synthesis withreversible dye terminators.

In some embodiments, the first set of sequence features includes aplurality of features based on mRNA expression values for the canceroustissue of the subject. That is, in some embodiments, a first pluralityof sequence reads is obtained from sequencing RNA molecules in a biopsyof the cancerous tissue of the subject, and features are formed based onexpression values for mRNA species. In some embodiments, these featuresinclude abundance values for individual mRNA species, e.g., relativeexpression values for individual genes, which may be normalized and/orbias corrected relative to raw abundance values obtained for each mRNAspecies, as described herein. In some embodiments, these featuresinclude arithmetic combinations of abundance values for more than onemRNA species, such as dimension-reduction component values. Forinstance, as illustrated in FIG. 17, in some embodiments, the first setof sequence features includes a set of nucleic acid-based featurescomprising RNA expression for the subject.

Generally, any methodology for measuring RNA expression (e.g.,determining an mRNA expression profile) can be used in conjunction withthe methods described herein, including nucleic acid sequencingmethodologies (e.g., of mRNA molecules directly or cDNA generated frommRNA), comparative hybridization (e.g., using microarrays), and/orquantitative PCR. In some embodiments, nucleic acid sequencing providesadvantages over other RNA expression profiling methodologies, such asthe ability to generate other types of genomic information from the samereaction, for example, genomic variant information (e.g., SNPs, MNPs,indels, small inversions, translocations, etc.).

Methods for RNA sequencing are well known in the art. For example,methods of RNA-seq for use in accordance with the methods describedherein are disclosed in Nagalakshmi et al., 2008, Science 320,1344-1349; and Finotell and Camillo, 2014, Briefings in FunctionalGenomics 14(2), 130-142, each of which is hereby incorporated herein byreference in its entirety. In some embodiments, RNA sequencing isperformed by whole exome sequencing (WES), as disclosed in, for example,Serrati et al., 2016, Onco Targets Ther. 9, 7355-7365 and Cieslik, M. etal., 2015 Genome Res. 25, 1372-81, each of which is hereby incorporatedherein by reference in its entirety, for all purposes. In someembodiments, the RNA sequencing is performed by nanopore sequencing. Areview of the use of nanopore sequencing techniques on the human genomecan be found in Jain et al., 2018, Nature 36(4), 338-345.

Methods for comparative hybridization (e.g., microarrays) are also knownin the art. Such methods are disclosed in Wang et al., 2009, Nat RevGenet 10, 57-63; Roy et al., 2011, Brief Funct Genomic 10:135-150;Shendure, 2008, Nat Methods 5, 585 -587; Cloonan et al., 2008, “Stemcell transcriptome profiling via massive-scale mRNA sequencing,” NatMethods 5, 613-619; Mortazavi et al., 2008, “Mapping and quantifyingmammalian transcriptomes by RNA-Seq,” Nat Methods 5, 621-628; andBullard et al., 2010, “Evaluation of statistical methods fornormalization and differential expression in mRNA-Seq experiments,” BMCBioinformatics 11, p. 94, each of which is hereby incorporated herein byreference in its entirety.

This list is not exhaustive of the RNA sequencing methods that may beused in accordance with the methods described herein. In someembodiments, the RNA sequencing is performed according to any one ormore sequencing methods known in the art. See, for example, a review ofRNA sequencing methods disclosed Kukurba et al., 2015, Cold Spring HarbProtoc. 11: 951-969, which is hereby incorporated herein by reference inits entirety.

In some embodiments, the first plurality of sequence reads obtained fromthe sequencing of the biopsy, e.g., RNA sequence reads, includes atleast 1000, at least 5000, at least 10,000, at least 50,000, at least100,000, at least 500,000, at least 1 million, at least 5 million, atleast 10 million, or more sequence reads. In some embodiments, the firstplurality of sequence reads is no more than 50,000, no more than100,000, no more than 500,000, no more than 1 million, no more than 5million, no more than 10 million, or no more than 50 million sequencereads. In some embodiments, the first plurality of sequence reads isfrom 1000 sequence reads to 100 million sequence reads, from 10,000sequence reads to 50 million sequence reads, or from 100,000 sequencereads to 50 million sequence reads. In some embodiments, the firstplurality of sequence reads falls within another range starting no lowerthan 1000 sequence reads and ending no higher than 500 million sequencereads.

In some embodiments, a measure of central tendency (e.g., mean, median,etc.) for the length of the sequence reads in the first plurality ofsequence reads is at least 15 nucleotides, at least 50 nucleotides, atleast 100 nucleotides, at least 250 nucleotides, at least 500nucleotides, at least 1000 nucleotides, or longer.

In some embodiments, the first plurality of sequence reads is mapped(e.g., to a reference sequence), quantified, normalized, scaled,deconvoluted, cleaned, filtered and/or otherwise preprocessed foranalysis. For example, in some embodiments, sequence reads are mapped toa reference construct (e.g., a reference exome or genome comprising aplurality of target gene sequences, or a similar reference map, e.g.,representing divergent sequences at particular loci across a species).In some embodiments, the number of sequence reads aligned to each regionof the reference construct (e.g., each gene) gives a measure of itslevel of expression (e.g., abundance). In some embodiments, the mappingis an alignment (e.g., an alignment performed via STAR and/or a k-merhashing-based pseudoalignment performed via Kallisto). In someembodiments, mRNA expression levels are normalized (e.g., to correct forGC content, sequencing depth, and/or gene or transcript length). Forexample, in some embodiments, raw read counts are quantile-normalized.In some embodiments, RNA expression levels are normalized relative tothe total number of transcripts in the first plurality of sequence readsand presented as transcripts per million. Methods for preprocessingsequence reads, including mapping raw RNA sequence reads to thetranscriptome, quantifying gene counts, normalization of gene counts,and/or deconvolution are known in the art, as described in U.S. PatentApplication No. 62/735,349, entitled “Methods of Normalizing andCorrecting RNA Expression Data,” filed on Sep. 24, 2018, and in U.S.patent application Ser. No. 16/732,229, entitled “TranscriptomeDeconvolution of Metastatic Tissue Samples,” filed on Dec. 31, 2019,each of which is hereby incorporated herein by reference in itsentirety.

In some embodiments, the first set of sequence features from the biopsyof the subject includes relative RNA abundance values (e.g., mRNAexpression values normalized to account for one or more of, e.g., atotal number of transcripts in a sample, a GC content of thetranscripts, length of the transcripts, etc.) for a plurality of genes.In some embodiments, one or more sequence features is a combination ofrelative RNA abundance values for two or more of the genes, e.g., alinear or non-linear combination. In some embodiments, one or moresequence features is a dimension reduction component value based onrelative RNA abundance values for two or more of the genes. In someembodiments, each sequence feature in the first set of sequence featuresfor the subject is an abundance value (e.g., a raw abundance value,normalized abundance value, standardized abundance value, scaledabundance value, or a combination thereof) for a single gene.

Generally, a sequence feature based on one or more RNA abundance values(e.g., mRNA abundance values) may take a variety of forms, includingwithout limitation, an absolute expression value (e.g., transcriptnumber), a relative expression value (e.g., relative fluorescent units,transcriptome analysis, and/or gene set expression analysis (GSEA)), acompound or aggregated expression value, a transformed expression value(e.g., log 2 and/or log 10 transformed), a change (e.g., fold- orlog-change) relative to a reference (e.g., a normal tissue sample of thesubject, a tumor-normal matched sample, a tissue sample from a healthysubject, a reference dataset comprising expression values from a cohortof healthy subjects, a housekeeping gene, and/or a reference standard),a measure of central tendency (e.g., mean, median, mode, weighted mean,weighted median, and/or weighted mode), a measure of dispersion (e.g.,variance, standard deviation, and/or standard error), an adjustedexpression value (e.g., normalized, scaled, and/or error-corrected),and/or a dimension-reduced expression value (e.g., principal componentvectors and/or latent components). Methods for obtaining sequencefeatures using dimension reduction techniques are known in the art,including but not limited to principal component analysis, factoranalysis, linear discriminant analysis, multi-dimensional scaling,isometric feature mapping, locally linear embedding, hessianeigenmapping, spectral embedding, t-distributed stochastic neighborembedding, and/or any substitutions, additions, deletions, modification,and/or combinations thereof as will be apparent to one skilled in theart. See, for example, Sumithra et al., 2015, “A Review of VariousLinear and Non Linear Dimensionality Reduction Techniques,” Int J CompSci and Inf Tech, 6(3), 2354-2360, which is hereby incorporated hereinby reference in its entirety. Other methods for obtaining abundancevalues for gene expression are contemplated, as disclosed in U.S.Provisional Patent Application No. 63/007,874, entitled “PredictingLikelihood and Site of Metastasis from Patient Records,” filed on Apr.9, 2020, which is incorporated by reference herein in its entirety.

In some embodiments, the first set of sequence features includesrelative abundance values for the expression of a plurality of genes. Insome embodiments, the first set of features includes relative abundancevalues for the expression of at least 25 genes. In some embodiments, thefirst set of features includes relative abundance values for theexpression of at least 50 genes. In some embodiments, the first set offeatures includes relative abundance values for the expression of atleast 75 genes. In some embodiments, the first set of features includesrelative abundance values for the expression of at least 100 genes. Insome embodiments, the first set of features includes relative abundancevalues for the expression of at least 250 genes. In some embodiments,the first set of features includes relative abundance values for theexpression of at least 150 genes, at least 200 genes, at least 300genes, at least 400 genes, at least 500 genes, at least 750 genes, atleast 1000 genes, or more.

In some embodiments, the first set of features includes relativeabundance values for the expression of no more than 4000 genes. In someembodiments, the first set of features includes relative abundancevalues for the expression of no more than 3000 genes. In someembodiments, the first set of features includes relative abundancevalues for the expression of no more than 2000 genes. In someembodiments, the first set of features includes relative abundancevalues for the expression of no more than 1000 genes. In someembodiments, the first set of features includes relative abundancevalues for the expression of no more than 900 genes, no more than 800genes, no more than 700 genes, no more than 600 genes, no more than 500genes, no more than 400 genes, no more than 250 genes. In someembodiments, the plurality of genes is no more than 1000 genes, no morethan 750 genes, no more than 500 genes, no more than 400 genes, no morethan 300 genes, no more than 250 genes, no more than 200 genes, no morethan 175 genes, no more than 150 genes, no more than 125 genes, no morethan 100 genes, no more than 75 genes, no more than 50 genes, no morethan 40 genes, or less.

In some embodiments, the first set of features includes relativeabundance values for the expression of from 25 to 4000 genes. In someembodiments, the first set of features includes relative abundancevalues for the expression of from 50 to 2000 genes. In some embodiments,the first set of features includes relative abundance values for theexpression of from 50 to 1000 genes. In some embodiments, the first setof features includes relative abundance values for the expression offrom 75 to 500 genes. In some embodiments, the first set of featuresincludes relative abundance values for the expression of another rangestarting no lower than 25 genes and ending no higher than 4000 genes.

In some embodiments, the first set of features includes relativeabundance values for the expression of at least 5, at least 10, at least20, at least 25, at least 30, at least 40, at least 50, at least 75, orall of the genes disclosed herein in the above sections (see,“Prediction of Colorectal Metastasis based on RNA”).

In some embodiments, the plurality of genes includes at least 20 of thegenes listed in Table 2. In some embodiments, the plurality of genesincludes at least 20, at least 25, at least 30, at least 35, at least40, at least 50, at least 75, at least 100, at least 125, at least 150,at least 175, at least 200, at least 250, at least 300, at least 350, atleast 400, at least 450, or all of the genes listed in Table 2. In someembodiments, the plurality of genes includes at least 50%, at least 55%,at least 60%, at least 70%, at least 75%, at least 80%, at least 85%, atleast 90%, at least 95%, or all of the genes listed in Table 2.

In some embodiments, the plurality of genes includes at least 20 of thegenes listed in Table 3. In some embodiments, the plurality of genesincludes at least 20, at least 25, at least 30, at least 35, at least40, at least 50, at least 75, at least 100, at least 125, at least 150,at least 175, at least 200, at least 250, at least 300, at least 350, atleast 400, at least 450, or all of the genes listed in Table 3.

In some embodiments, the plurality of genes includes at least 20 of thegenes listed in Table 3 that were used in at least 150 iterations of themodel. In some embodiments, the plurality of genes includes at least 20,at least 25, at least 30, at least 35, at least 40, at least 50, atleast 75, at least 100, at least 125, at least 150, at least 175, atleast 200, at least 250, at least 300, at least 350, at least 400, atleast 450, or all of the genes listed in Table 3 that were used in atleast 150 iterations of the model. In some embodiments, the plurality ofgenes includes at least 50%, at least 55%, at least 60%, at least 70%,at least 75%, at least 80%, at least 85%, at least 90%, at least 95%, ofall of the genes listed in Table 3 that were used in at least 150iterations of the model.

In some embodiments, the plurality of genes includes at least 20 of thegenes listed in Table 3 that were used in at least 160 iterations of themodel. In some embodiments, the plurality of genes includes at least 20,at least 25, at least 30, at least 35, at least 40, at least 50, atleast 75, at least 100, at least 125, at least 150, at least 175, atleast 200, at least 250, at least 300, at least 350, at least 400, atleast 450, or all of the genes listed in Table 3 that were used in atleast 160 iterations of the model. In some embodiments, the plurality ofgenes includes at least 50%, at least 55%, at least 60%, at least 70%,at least 75%, at least 80%, at least 85%, at least 90%, at least 95%, ofall of the genes listed in Table 3 that were used in at least 160iterations of the model.

In some embodiments, the plurality of genes includes at least 20 of thegenes listed in Table 3 that were used in at least 165 iterations of themodel. In some embodiments, the plurality of genes includes at least 20,at least 25, at least 30, at least 35, at least 40, at least 50, atleast 75, at least 100, at least 125, at least 150, at least 175, atleast 200, at least 250, at least 300, at least 350, at least 400, atleast 450, or all of the genes listed in Table 3 that were used in atleast 165 iterations of the model. In some embodiments, the plurality ofgenes includes at least 50%, at least 55%, at least 60%, at least 70%,at least 75%, at least 80%, at least 85%, at least 90%, at least 95%, ofall of the genes listed in Table 3 that were used in at least 165iterations of the model.

In some embodiments, the plurality of genes includes at least 20 of thegenes listed in Table 3 that were used in at least 165 iterations of themodel. In some embodiments, the plurality of genes includes at least 20,at least 25, at least 30, at least 35, at least 40, at least 50, atleast 75, at least 100, at least 125, at least 150, at least 175, atleast 200, at least 250, at least 300, at least 350, at least 400, atleast 450, or all of the genes listed in Table 3 that were used in atleast 165 iterations of the model. In some embodiments, the plurality ofgenes includes at least 50%, at least 55%, at least 60%, at least 70%,at least 75%, at least 80%, at least 85%, at least 90%, at least 95%, ofall of the genes listed in Table 3 that were used in at least 165iterations of the model.

In some embodiments, the plurality of genes includes at least 20 of thegenes listed in Table 3 that were used in at least 171 iterations of themodel. In some embodiments, the plurality of genes includes at least 20,at least 25, at least 30, at least 35, at least 40, at least 50, atleast 75, at least 100, at least 125, at least 150, at least 175, atleast 200, at least 250, at least 300, at least 350, at least 400, atleast 450, or all of the genes listed in Table 3 that were used in atleast 171 iterations of the model. In some embodiments, the plurality ofgenes includes at least 50%, at least 55%, at least 60%, at least 70%,at least 75%, at least 80%, at least 85%, at least 90%, at least 95%, ofall of the genes listed in Table 3 that were used in at least 171iterations of the model.

In some embodiments, the plurality of genes includes at least 20 of thegenes listed in Table 3 that were used in at least 172 iterations of themodel. In some embodiments, the plurality of genes includes at least 20,at least 25, at least 30, at least 35, at least 40, at least 50, atleast 75, at least 100, at least 125, at least 150, at least 175, atleast 200, at least 250, at least 300, at least 350, at least 400, atleast 450, or all of the genes listed in Table 3 that were used in atleast 172 iterations of the model. In some embodiments, the plurality ofgenes includes at least 50%, at least 55%, at least 60%, at least 70%,at least 75%, at least 80%, at least 85%, at least 90%, at least 95%, ofall of the genes listed in Table 3 that were used in at least 172iterations of the model.

In some embodiments, the plurality of genes includes at least 20 of thegenes listed in Table 3 that were used in all 173 iterations of themodel. In some embodiments, the plurality of genes includes at least 20,at least 25, at least 30, at least 35, at least 40, at least 50, atleast 75, at least 100, at least 125, at least 150, at least 175, atleast 200, at least 250, at least 300, at least 350, at least 400, atleast 450, or all of the genes listed in Table 3 that were used in all173 iterations of the model. In some embodiments, the plurality of genesincludes at least 50%, at least 55%, at least 60%, at least 70%, atleast 75%, at least 80%, at least 85%, at least 90%, at least 95%, ofall of the genes listed in Table 3 that were used in all 173 iterationsof the model.

In some embodiments, the first set of sequence features includes aplurality of dimension reduction component values determined fromrelative abundance values for a plurality of genes. For instance,Example 1 describes an instance where relative expression values for the500 most correlated genes are used to generate 40 sequence featuresusing singular value decomposition, which are the basis for a predictionmodel. Accordingly, in some embodiments, the first set of sequencefeatures include a plurality of dimension reduction component valuesdetermined from relative abundance values for at least 25, at least 50,at least 75, at least 100, at least 150, at least 200, at least 250, atleast 500, at least 750, at least 1000, or more genes. In someembodiments, the first set of sequence features include a plurality ofdimension reduction component values determined from relative abundancevalues for no more than 2000 genes, no more than 1500 genes, no morethan 1000 genes, no more than 750 genes, no more than 500 genes, orless. In some embodiments, the first set of sequence features include aplurality of dimension reduction component values determined fromrelative abundance values for from 25 to 2000 genes, from 50 to 1000genes, or from 100 to 750 genes.

Thus, for example, referring again to FIG. 16, in some embodiments themethod includes obtaining a plurality of data elements for the subject'scancer, where the plurality of data elements includes a first set ofsequence features comprising relative abundance values for theexpression of a plurality of genes (e.g., at least 30 genes) in a biopsyof the cancer obtained from the subject. In some embodiments, obtainingthe first set of sequence features includes obtaining a plurality of atleast 10,000 sequence reads, where the plurality of sequence reads isobtained from a plurality of RNA molecules from the biopsy of the cancerobtained from the subject, and determining, from the plurality ofsequence reads, relative abundance values for the plurality of genes.For example, in some embodiments, the determining includes mapping(e.g., alignment via STAR and/or pseudoalignment via Kallisto) theplurality of at least 10,000 sequence reads to a reference constructcomprising a plurality of target gene sequences (e.g., mRNA sequences)and counting the number of sequence reads aligned to each gene ofinterest. In some embodiments, the determining further includesnormalizing the plurality of sequence reads (e.g., for GC content,transcript length, library depth, etc.).

In some embodiments, the plurality of data elements further includesadditional sequence features, including but not limited to RNA-basedfeatures (including mRNA and derivatives of RNA such as cDNA and/orproteins), DNA-based features, tumor purity features, germline genomicsfeatures, cancer genomics features, transcript (splice) isoforms,expression values for gene copies (e.g., a wild-type copy of a gene, avariant copy of the gene, or an arithmetic combination thereof),epigenetic features, methylation-based features, homologousrecombination deficiency status, microsatellite stability status,microsatellite repeat length, tumor mutation burden, SNPs, SNVs, indels,fusions, copy number variations, amplifications, deletions, variantallelic ratios, genomic marker status (e.g., associated with anactionable therapy), pathway features (e.g., RNA pathway features, geneset enrichment analysis, functional pathway features, etc.), and/or anycombination thereof. In some embodiments, sequence features are obtainedby any sequencing method (e.g., RNAseq, mRNA sequencing, DNA sequencing,microarrays, etc.) and sequencing processing methods (e.g.,normalization, error-correction, deconvolution, etc.) disclosed herein.As illustrated in FIG. 17, in some instances, the plurality of dataelements includes additional nucleic acid-based features such as tumorpurity, germline genomics, and/or cancer genomics features. In someembodiments, the first set of sequence features includes any of thefeatures described in the feature modules disclosed in the abovesections (see, “Generating and Modeling Predictions of PatientObjectives”).

For example, in some embodiments, the plurality of data elements furtherincludes a mutational status for one or more genes in the genome of thecancer. In some embodiments, the plurality of data elements furtherincludes a mutational status for one or more genes in the genome of anon-cancerous tissue of the subject. In some embodiments, the pluralityof data elements further includes a copy number status for one or moregenomic regions (e.g., one or more genes) of the cancer.

In some embodiments, the plurality of data elements further includes asingle-sample gene set enrichment analysis (ssGSEA) score for thetranscriptional profile of the cancer. Methods for obtaining ssGSEAscores are described in the above sections (see, “Generating andModeling Predictions of Patient Objectives”) and in Subramanian et al.,2005, PNAS 102, 15545-15550 and Liberzon et al., 2015, Cell Systems 23;1(6): 417-425, each of which is hereby incorporated herein by referencein its entirety.

Other methods for obtaining sequence features are contemplated, asdisclosed in U.S. Provisional Patent Application No. 63/007,874,entitled “Predicting Likelihood and Site of Metastasis from PatientRecords,” filed on Apr. 9, 2020, which is incorporated by referenceherein in its entirety.

Selection of sequence features. In some embodiments, the first set ofsequence features is selected using a first training plurality ofsequence reads (e.g., a training dataset), where the first trainingplurality of sequence reads is obtained from one or more samplesobtained from each respective training subject in a first plurality oftraining subjects. In some embodiments, a set of selected sequencefeatures (e.g., a panel of genes of interest) and any correspondingvalues for the set of selected sequence features (e.g., gene expressionvalues) obtained from the training dataset are used as input fortraining a model (e.g., a set of models). In some embodiments, the firsttraining plurality of sequence reads is obtained by a sequencing methodand/or preprocessed by, for example, any of the methods for obtainingthe first set of sequence features disclosed herein.

In some embodiments, feature selection is performed based on theinformative quality of the respective features. For example, asdescribed below, in some embodiments a feature is considered“informative” if it exhibits a variable characteristic that can be usedto predict metastasis of a cancer in a subject. For instance, in someembodiments, a feature is considered informative if it has at least athreshold variance (e.g., statistical variance and/or standarddeviation) or a threshold level of abundance (e.g., absolute or relativeabundance, differential gene expression, and/or fold change). In someembodiments, a feature is considered “uninformative” if it does notexhibit a characteristic that can be used to predict metastasis of acancer in a subject.

In some embodiments, feature selection is performed using one or moremodels, including any of the models disclosed in the above sections(see, “Generating and Modeling Predictions of Patient Objectives”).Feature selection is performed by generating predictions using thetraining dataset and evaluating (e.g., scoring) the importance offeatures based on the generated predictions, thus identifying featuresthat drive and/or are correlated with such predictions. For example, insome embodiments, a feature selection method is performed by executing aprediction generation more than 100 times on a classification model,each time with a different assignment of cross-validation folds and holdout set. This process results in over 100 out-of-fold cross validatedscores on the training set and over 100 of hold-out (or test set)scores. A distribution of performance metrics is generated based on thescores of the predictions. By assessing the distribution of performancemetrics in the over 100 different training subsets, features thatgenerate high-scoring predictions across a high percentage of trainingruns and performance metrics can be identified, rather relying on singlepoint estimates (which can have a large degree of variance). Suchmethods allow for more robust feature importance evaluation and improvedfeature selection. Other methods for feature selection are known in theart, as reviewed, for example, in Saeys et al., 2007, “A review offeature selection techniques in bioinformatics,” Bioinformatics 23:19,2507-2517; doi: 10.1093/bioinformatics/btm344, which is herebyincorporated herein by reference in its entirety.

Personal characteristics 1608. In some embodiments, the plurality ofdata elements includes one or more personal characteristics (1608) aboutthe subject (e.g., stored in an optional personal characteristicsconstruct 1530) selected from the group consisting of age, gender,and/or race.

In some embodiments, as illustrated in FIG. 17, the one or more personalcharacteristics includes, but is not limited to, age, gender, race,personal habits (e.g., smoking, drinking, diet, etc.), weight,biological statuses (e.g., high blood pressure, dry skin, other diseasesand/or other pertinent medical conditions), and/or other demographicfeatures. In some embodiments, the one or more personal characteristicsinclude smoking status and/or menopausal status. In some embodiments,the one or more personal characteristics include one or more demographicfeatures that is associated with the respective cancer condition of thesubject (e.g., a smoking status and/or a menopausal status inconjunction with a lung cancer, breast cancer, and/or an ovarian cancer.Other, non-limiting examples of personal characteristics that may beused in the disclosed metastasis models are described above in thesection titled “Generating and Modeling Predictions of PatientObjectives.”

Clinical features 1610. In some embodiments, the plurality of dataelements includes one or more clinical features related to the diagnosisor treatment of the cancer in the subject (e.g., stored in an optionalclinical features construct 1532).

In some embodiments, the one or more clinical features related to thediagnosis or treatment of the cancer in the subject is selected from thegroup consisting of a stage of the cancer, grading of the cancer, ahistopathological grade of the cancer, presence and/or absence ofcancer, a cancer type, a cancer subtype, a tissue of origin, a therapyadministered to the subject, a symptom associated with cancer ormetastasis thereof, and a comorbidity with the cancer. Other,non-limiting examples of clinical features that may be used in thedisclosed metastasis models are described in FIGS. 4, 5, 11, 12, and 17,and discussed in detail above in the section titled “Generating andModeling Predictions of Patient Objectives.”

Temporal features 1611. In some embodiments, the plurality of dataelements includes one or more temporal features related to the diagnosisor treatment of the cancer in the subject (e.g., stored in an optionalclinical features construct 1532).

In some embodiments, the one or more clinical features includes the oneor more temporal features. In some embodiments, the one or more temporalfeatures is any temporal feature associated with any clinical feature inthe one or more clinical features. In some embodiments, the one or moretemporal features includes a first temporal element indicating theduration of time since a diagnosis for the cancer (e.g., a diagnosis ofthe cancer, including presence, absence, stage, type, subtype, grade,histopathological grade, and/or tissue of origin), a second temporalelement indicating the duration of time since an administration of thetherapy (e.g., since first administration, since last administration,and/or since completion of a first therapeutic regimen), a thirdtemporal element indicating the duration of time since an experience ofthe symptom (e.g., since first experience of the symptom and/or sincelast experience of the symptom), and a fourth temporal elementindicating the duration of time since an experience of the comorbidity(e.g., since the beginning of the comorbidity and/or since theresolution of the comorbidity). Other, non-limiting examples of clinicalfeatures that may be used in the disclosed metastasis models aredescribed in FIGS. 4, 5, 11, 12, and 17, and discussed in detail abovein the section titled “Generating and Modeling Predictions of PatientObjectives.”

Additional data elements. In some embodiments, the plurality of dataelements further includes one or more pathological features, one or moreviral features, one or more metabolomic features, one or more microbiomefeatures, and/or any combination thereof.

For example, in some embodiments, the plurality of data elements furtherincludes one or more physical characteristics of the biopsy of thecancer. In some embodiments, the one or more physical characteristics ofthe biopsy of the cancer are obtained from a biological imaging of thebiopsy of the cancer. In some embodiments, the biological image includesbiologically meaningful features and geometrically meaningful features,both of which can be included as features in the plurality of dataelements.

For example, in some embodiments, the one or more physicalcharacteristics of a biopsy of a cancer includes a tumor percentage(e.g., a percentage of the detected tissue area on the slide classifiedas tumor, e.g., indicated as a value between 0 and 100).

In some embodiments, the one or more physical characteristics includes atumor cell percentage (e.g., a percentage of the total cells that aretumor cells, as opposed to lymphocytes, e.g., indicated as a valuebetween 0 and 100).

In some embodiments, the one or more physical characteristics includes atumor infiltrating lymphocytes percentage (e.g., calculated as totalnumber of lymphocytes within the tumor region divided by total numbercells in the tumor region, e.g., indicated as a value between 0 and100).

In some embodiments, the one or more physical characteristics includestumor budding features (e.g., for colorectal cancer. In someembodiments, tumor budding features are represented as integer countsfor a number of detected tumor buds. In some embodiments, tumor buddingfeatures are normalized by density within a given area.

In some embodiments, geometrically meaningful features includeaggregation metrics (e.g., a minimum, average, median, and/or maximum)of tumor perimeter. For example, in some embodiments, geometricallymeaningful features are used to measure the perimeter of a tumor biopsywithin a slide, given in pixels, where each pixel has fixed dimensions(e.g., 8 um by 8 um).

In some embodiments, geometrically meaningful features include averagetumor cell circularity (e.g., calculated as cell area divided by thesquare of the perimeter, averaged over all cells, e.g., indicated as avalue ranging between 0 and 1).

In some embodiments, geometrically meaningful features include averagetumor cell length and/or aspect ratio. In some such embodiments, aspectratio is determined by calculating the eigenvalues for the cell shape ofa cell in the tumor biopsy, giving relative, rotation-independent valuesfor the length and width of the cell. Specifically, all pixels that areassociated with the cell are first identified, using the (x,y) pixellocations as points for a covariance matrix. Eigenvalues are thencalculated, and the first component is taken as the length of the cell.The aspect ratio is calculated by dividing the second eigenvalues by thefirst eigenvalues. In some embodiments, the aspect ratio is indicated asa value ranging between 0 and 1.

Imaging features, including biologically meaningful features andgeometrically meaningful features, are further described in the abovesections of the present disclosure (see, “Generating and ModelingPredictions of Patient Objectives”).

Example inputs for models for predicting metastasis of a cancer in asubject are illustrated in FIGS. 17 and 18. In some embodiments, inputsinclude personal characteristics, clinical features (e.g., clinical dataand/or clinomic data), nucleic acid-based features (e.g., moleculardata), pathology features (e.g., pathologist evaluation), and/ortemporal data (e.g., time since clinical events as illustrated, forexample, in FIG. 7).

Additional features to be used as inputs for using a trained model(e.g., a predictive and/or classification model) to predict metastasisof a cancer in a subject are contemplated, as illustrated in FIGS. 4, 5,11 and 12 and further described in the above sections of the presentdisclosure (see, “Generating and Modeling Predictions of PatientObjectives”). In some embodiments, any of the features disclosed hereinare obtained from a feature collection 205, a feature module 110, afeature store 120, a feature selector 200, a feature generator 300,and/or a data structure 1522, as illustrated in FIGS. 1-3, 14, and 15and described in detail herein (see, “Generating and ModelingPredictions of Patient Objectives”). Suitable features also include anysubstitutions, additions, deletions, modifications, and/or combinationsof any of the embodiments disclosed herein, as will be apparent to oneskilled in the art.

Obtaining Indications of Metastasis.

One or more metastasis models are then applied to the data elementsidentified for the subject. In some embodiments, a single modelproviding an indication of whether the cancer will metastasize (e.g., toany tissue within the subject) is applied to the data elements. Forexample, in some embodiments, the one or more models is a single modelthat provides one or more indications (e.g., predictions) on whether acancer will metastasize, to any or an unspecified tissue, within asingle or an unspecified duration of time. The one or more models can beone or more models 1538 stored in a classification module 1536, as partof a system 1500 depicted in FIG. 15. Furthermore, the one or moreindications can be one or more indications 1542 stored in an outputmodule 1540 in the system 1500.

In other embodiments, multiple metastasis models are applied to the dataelements. For example, in some embodiments, separate models are appliedto provide an indication of whether the cancer will metastasize to eachof a plurality of tissues, that is, one or more model is applied foreach tissue of interest. Similarly, in some embodiments, separate modelsare applied to provide an indication of whether the cancer willmetastasize (to a specific tissue or at all) within each of a pluralityof time horizons, that is, one or more model is applied for each timehorizon of interest.

Accordingly, in some embodiments, referring to Block 1612, the methodfurther includes applying, to the plurality of data elements for thesubject's cancer, one or more models that are collectively trained toprovide a respective one or more indications of whether the cancer willmetastasize in the subject, thereby predicting whether the cancer willmetastasize. In some embodiments, the one or more indications of whetherthe cancer will metastasize in the subject predicts whether the cancerwill metastasize to any tissue in the subject (e.g., does not specifythe metastasis site). In some embodiments, the one or more indicationsof whether the cancer will metastasize in the subject predicts whetherthe cancer will metastasize at any time horizon (e.g., does not specifythe time period for metastasis). In some embodiments, the one or moreindications of whether the cancer will metastasize in the subjectpredicts whether the cancer will metastasize within a single timehorizon.

In some embodiments, the method further includes applying, to theplurality of data elements for the subject's cancer, a set of modelsthat are collectively trained to provide, for each respective tissue ina plurality of tissues, a respective set of indications of whether thecancer will metastasize to the respective tissue in the subject, whereinthe respective set of indications includes a respective indication foreach respective time horizon in a plurality of time horizons, therebydetermining a plurality of indications of whether the cancer willmetastasize comprising, for each respective tissue in the plurality oftissues, a respective set of indications comprising, for each respectivetime horizon in a plurality of time horizons, a respective indication ofwhether the cancer in the subject will metastasize to the respectivetissue within the respective time horizon.

Metastasis sites. In some embodiments, the method includes obtaining anindication of whether the cancer of the subject will metastasize to oneor more tissues in a plurality of tissues (e.g., one or more potentialmetastasis sites). In some embodiments, the indication is adetermination that the cancer will or will not metastasize (e.g., abinary indication), and/or a likelihood that the cancer will metastasize(e.g., a probability).

In some embodiments, the plurality of tissues (e.g., potentialmetastasis sites) includes lymph tissue (e.g., proximal lymph tissueand/or distal lymph tissue), liver tissue, and/or lung tissue. In someembodiments, the plurality of tissues includes two or more similartissue types found in different locations within the body (e.g.,proximal lymph tissue and distal lymph tissue).

In some embodiments, the plurality of tissues includes one or moretissues selected from cardiovascular (e.g., heart, blood, and/or bloodvessels), lymphatic (e.g., lymph, lymph nodes, and/or lymph vessels),digestive (e.g., mouth, salivary glands, esophagus, stomach, liver,gallbladder, exocrine pancreas, small intestine, and/or largeintestine), endocrine (e.g., pituitary, pineal, thyroid, parathyroids,endocrine pancreas, adrenals, testes, and/or ovaries), integumentary(e.g., skin, hair, and/or nails), muscular (e.g., skeletal, cardiac,and/or smooth muscles), nervous (e.g., brain, spinal cord, nerves,sensory organs, eyes, ears, tongue, skin, and/or nose), reproductive(e.g., fallopian tubes, uterus, vagina, ovaries, mammary glands, testes,vas deferens, seminal vesicles, prostate, and/or penis), respiratory(e.g., mouth, nose, pharynx, larynx, trachea, bronchi, lungs, and/ordiaphragm), skeletal (e.g., bones, cartilage, joints, tendons, and/orligaments, urinary (e.g., kidneys, ureters, urinary bladder, and/orurethra) and/or immune (e.g., leukocytes, tonsils, adenoids, thymus,and/or spleen).

In some embodiments, the plurality of tissues includes one or moretissues selected based on the cancer of the subject. For example, insome embodiments, the cancer of the subject is bladder, and theplurality of tissues includes bone, liver, and/or lung. In someembodiments, the cancer of the subject is breast, and the plurality oftissues includes bone, brain, liver, and/or lung. In some embodiments,the cancer of the subject is colon, and the plurality of tissuesincludes liver, lung, and/or peritoneum. In some embodiments, the cancerof the subject is kidney, and the plurality of tissues includes adrenalgland, bone, brain, liver, and/or lung. In some embodiments, the cancerof the subject is lung, and the plurality of tissues includes adrenalgland, bone, brain, liver, and/or lung. In some embodiments, the cancerof the subject is melanoma, and the plurality of tissues includes bone,brain, liver, lung, skin, and/or muscle. In some embodiments, the cancerof the subject is ovary, and the plurality of tissues includes liver,lung, and/or peritoneum. In some embodiments, the cancer of the subjectis pancreas, and the plurality of tissues includes liver, lung, and/orperitoneum. In some embodiments, the cancer of the subject is prostate,and the plurality of tissues includes adrenal gland, bone, liver, and/orlung. In some embodiments, the cancer of the subject is rectal, and theplurality of tissues includes liver, lung, and/or peritoneum. In someembodiments, the cancer of the subject is stomach, and the plurality oftissues includes liver, lung, and/or peritoneum. In some embodiments,the cancer of the subject is thyroid, and the plurality of tissuesincludes bone, liver, and/or lung. In some embodiments, the cancer ofthe subject is uterus, and the plurality of tissues includes bone,liver, lung, peritoneum, and/or vagina.

In some embodiments, the plurality of tissues includes at least 2, atleast 3, at least 4, at least 5, at least 6, at least 7, at least 8, atleast 9, at least 10, at least 15, at least 20, at least 30, at least40, at least 50, at least 60, at least 70, at least 80, at least 90, atleast 100, at least 125, at least 150, at least 175, or at least 200tissues (e.g., potential metastasis sites).

In some embodiments, the plurality of tissues includes no more than 200,no more than 175, no more than 150, no more than 125, no more than 100,no more than 80, no more than 60, no more than 50, no more than 40, nomore than 30, or no more than 20 tissues (e.g., metastasis sites). Insome embodiments, the plurality of tissues is from 2 and 200 tissues. Insome embodiments, the plurality of tissues is from 3 and 100 tissues,from 10 and 70 tissues, from 20 to 200 tissues, from 2 to 20 tissues, orfrom 50 to 100 tissues. In some embodiments, the plurality of tissuesfalls within another range starting no lower than 2 tissues and endingno higher than 200 tissues (e.g., potential metastasis sites).

Time horizons. In some embodiments, the method includes obtaining anindication of whether the cancer of the subject will metastasize in thesubject (e.g., to one or more tissues in the plurality of tissues)within a respective time horizon. For example, in some embodiments, themethod includes obtaining an indication (e.g., a prediction) that thecancer will metastasize within a given period of time within thesubject's clinical timeline (e.g., after an occurrence of a clinicalevent). In some embodiments, the method includes obtaining an indicationof whether the cancer of the subject will metastasize in the subject(e.g., to one or more tissues in the plurality of tissues) within arespective time horizon in a plurality of time horizons.

In some embodiments, a time horizon is a period of time (e.g., aduration of time). For example, in some embodiments a time horizon is aperiod of time comprising at least 1, at least 2, at least 3, or atleast 4 weeks. In some embodiments, a time horizon is a period of timecomprising at least 1, at least 2, at least 3, at least 4, at least 5,at least 6, at least 7, at least 8, at least 9, at least 10, at least11, or at least 12 months. In some embodiments, a time horizon is aperiod of time comprising at least 1, at least 2, at least 3, at least4, at least 5, at least 6, at least 7, at least 8, at least 9, at least10, at least 15, at least 20, at least 25, at least 30, at least 35, atleast 40, at least 50, at least 60, at least 70, at least 80, at least90, or at least 100 years. In some embodiments, a time horizon is about1, about 2, about 3, about 4, about 5, about 6, about 7, about 8, about9, about 10, about 11, or about 12 months. In some embodiments, a timehorizon is a period of time comprising about 1, about 2, about 3, about4, about 5, about 6, about 7, about 8, about 9, about 10, about 15,about 20, about 25, about 30, about 35, about 40, about 50, about 60,about 70, about 80, about 90, or about 100 years. In some embodiments, atime horizon is from 1 month to 3 months, from 1 month to 6 months, from3 months to 1 year, from 6 months to 5 years, from 1 year to 10 years,or from 2 years to 50 years.

In some embodiments, a time horizon in the plurality of time horizons isa period of time extending from a temporal element (e.g., a time since afirst diagnosis, a time since a subsequent diagnosis other than thefirst diagnosis, a time since first administration of a therapy, a timesince a last administration of a therapy, a time since a completion of afirst therapeutic regimen, a time since a first experience of a symptom,a time since a last experience of a symptom, a time since a firstexperience of a comorbidity, and/or a time since a resolution of acomorbidity).

In some embodiments, a time horizon is a period of time that is any ofthe above durations (e.g., weeks, months, and/or years) following any ofthe above temporal elements (e.g., any period of time starting from aclinical event in the subject's timeline). Examples of time horizons arefurther illustrated in FIG. 7 (e.g., “anchor point” indicates a startingtemporal element; “target window” indicates a time horizon following thestarting temporal element).

In some embodiments, the plurality of time horizons for a respective setof indications (e.g., for a respective tissue) includes at least 2, atleast 3, at least 4, at least 5, at least 6, at least 7, at least 8, atleast 9, at least 10, at least 15, at least 20, at least 25, at least30, at least 35, at least 40, at least 50, at least 60, at least 70, atleast 80, at least 90, or at least 100 time horizons (e.g., targetwindows). In some embodiments, the plurality of time horizons includesno more than 100, no more than 80, no more than 60, no more than 50, nomore than 40, no more than 30, no more than 20, no more than 15, or nomore than 10 time horizons. In some embodiments, the plurality of timehorizons is from 2 to 100 time horizons. In some embodiments, theplurality of time horizons is from 2 to 10, between 5 and 50, between 10and 20, or between 10 and 100 time horizons.

In some embodiments, each time horizon in the plurality of time horizonsincludes the same duration of time (e.g., 6 months). In someembodiments, a first time horizon in the plurality of time horizonsincludes a different duration of time from a second time horizon in theplurality of time horizons (e.g., 6 months and 12 months). In someembodiments, a first time horizon overlaps with a second time horizon inthe plurality of time horizons. For example, in some embodiments, theplurality of time horizons includes a first time horizon of 6 monthsafter a first temporal element (e.g., a cancer diagnosis) and a secondtime horizon of 12 months after the respective first temporal element,where the first 6 months in the second time horizon of 12 monthsoverlaps with the first time horizon of 6 months. As another example, insome embodiments, the plurality of time horizons includes a first timehorizon spanning the period between months 6 and 18 after a temporalelement (e.g., a cancer diagnosis), and a second time horizon spanningthe period between months 12 and 24 after the respective temporalelement. In some embodiments, a first time horizon in the plurality oftime horizons does not overlap with a second time horizon in theplurality of time horizons (e.g., each time horizon encompasses adistinct epoch of time (e.g., months 6-12, years 1-2, years 2-3, years3-6, etc.)).

In some embodiments, the plurality of time horizons for each respectiveset of indications in a plurality of sets of indications (e.g., for eachrespective tissue in a plurality of tissues) includes the same numberand/or durations of time horizons (e.g., such that indications areprovided for each tissue in a plurality of tissues across a uniform setof time horizons). In some embodiments, the plurality of time horizonsfor each respective set of indications in a plurality of sets ofindications (e.g., for each respective tissue in a plurality of tissues)includes different numbers and/or durations of time horizons.

In some embodiments, the number and/or duration of time horizons in theplurality of time horizons is predetermined (e.g., fixed). In someembodiments, each time horizon in the plurality of time horizons isselectable and/or adjustable by a user or practitioner (e.g., such thatindications are provided for any desired target window and/or any pointof interest in a subject's clinical timeline). In some embodiments, theselection and/or adjustment of time horizons is performed via a useraffordance (e.g., a typed-in command and/or a scrollable bar in aninteractive graphical representation).

Indications. In some embodiments, predictions of cancer metastasisprovide information on whether a cancer will metastasize, to any or anunspecified tissue, within a single or an unspecified duration of time(e.g., as one or more indications of metastasis 1542 provided by acorresponding one or more models 1538).

Additional example predictions of cancer metastasis are furtherillustrated with reference to FIGS. 17 and 18. For example, FIG. 18illustrates that an indication of cancer metastasis, in someembodiments, provides information on the location to which the cancerwill metastasize (e.g., predicted location, within a single orunspecified duration of time). In some embodiments, an indication ofcancer metastasis provides information on the time until metastasis isexpected to occur in the subject (e.g., predicted time horizon, to anyand/or unspecified tissue). In some embodiments, an indication of cancermetastasis further provides information on a specific location to whichthe cancer will metastasize by a specific time in the future (e.g.,specific tissue and time horizon). FIG. 17 further illustrates anexample of a plurality of indications of whether the cancer willmetastasize (e.g., where the plurality of indications includes, for eachrespective tissue in the plurality of tissues, a respective set ofindications for each respective time horizon in the plurality of timehorizons). Thus, for example, for a plurality of N tissues S and aplurality of M time horizons H, the plurality of indications can includeall possible combinations of N tissues and M time horizons from S₁H₁ toS_(N)H_(M). In some embodiments, the plurality of indications includesseparate indications for similar tissue types found in differentlocations within the body (e.g., proximal lymph tissue and distal lymphtissue).

In some embodiments, an indication of whether the cancer of the subjectwill metastasize is a binary output (e.g., “yes” or “no” and/or “likely”or “not likely”). In some embodiments, an indication of whether thecancer of the subject will metastasize is a likelihood or a probability(e.g., a value between 0 and 1 and/or a percentage between 0 and 100).In some embodiments, the indication is transformed and/or scaled (e.g.,from a percentage to a probability).

In some embodiments, the one or more models (e.g., comprising predictiveand/or classification models) provides both a binary and a non-binaryindication of whether the cancer of the subject will metastasize. Forexample, in some embodiments, an output from the one or more modelsincludes a probability value between 0 and 1 and a binary output basedon an interpretation of the probability value (e.g., whether theprobability value satisfies a likelihood cutoff threshold). Thus, if theone or more models provides a probability of 0.98 that the cancer willmetastasize (e.g., to a respective tissue within a respective timehorizon, or to any tissue at any time), and the probability thresholdfor a positive indication is 0.95, then the one or more models wouldprovide both the probability (0.98) and a binary indication (“yes”). Insome embodiments, the likelihood cutoff threshold (e.g., the probabilitythreshold for a positive indication) is a hyperparameter. In someembodiments, a hyperparameter is predetermined. In some embodiments, ahyperparameter is provided by a user or practitioner. In someembodiments, a hyperparameter is adjustable and/or is inputted by a useror practitioner. In some embodiments, a hyperparameter is optimizedbased on one or more optimization analyses. In some embodiments,optimization of a hyperparameter is performed to increase or decreasethe stringency of the classification.

In some embodiments, where the one or more indications is a plurality ofindications corresponding to a plurality of time horizons and/or aplurality of tissues, an indication of whether the cancer of the subjectwill metastasize includes a ranking for a respective tissue and/or arespective time horizon (e.g., of a ranked likelihood that the cancerwill metastasize to the respective tissue and/or within the respectivetime horizon). For example, in some embodiments, the plurality ofindications includes, for each respective tissue in the plurality oftissues, a ranking of the likelihood that the cancer will metastasize tothe respective tissue within each respective time horizon in theplurality of time horizons. In some embodiments, the ranking isperformed by comparing between all possible combinations of tissues andtime horizons (e.g., from S₁H₁ to S_(N)H_(M)) in order to generate aranked list.

In some embodiments, the one or more models further provide a confidencemeasure for the one or more indications of whether the cancer of thesubject will metastasize. In some embodiments, the confidence measure isa p-value, a confidence interval, a standard deviation, a variance, astandard error, and/or a distribution.

In some embodiments, each indication in the one or more indications(e.g., for a respective one or more combinations of possible timehorizons and/or tissues) are the same or different.

In some embodiments, each model in the one or more model provides adifferent indication in the corresponding one or more indications (e.g.,each respective model in the one or more models provides a respectiveindication in the one or more indications). In some embodiments, the oneor more models collectively provide the one or more indications. In someembodiments, the one or more models collectively provide a singleindication. In some embodiments, the one or more models is a singlemodel that provides a single indication.

Models. In some embodiments, a model in the one or more models 1538(e.g., set of models) is a predictive model and/or a classificationmodel.

In some embodiments, a model in the one or more models includes analgorithm selected from the group consisting of a neural networkalgorithm, a support vector machine algorithm, a Naive Bayes algorithm,a nearest neighbor algorithm, a boosted regression decision treeensemble algorithm, a random forest decision tree ensemble algorithm,and a multinomial logistic regression algorithm. In some embodiments,the model is a multinomial logistic regression algorithm comprising aregularization parameter (e.g., an L1 or an L2 (e.g., ridge)regularization penalty).

Logistic regression classifiers are disclosed in Agresti, AnIntroduction to Categorical Data Analysis, 1996, Chapter 5, pp. 103-144,John Wiley & Son, New York, which is hereby incorporated by reference.In some embodiments, the logistic regression classifier includes atleast 10, at least 20, at least 50, at least 100 weights, or at least1000 weights and requires a computer to calculate because it cannot bementally solved.

A k-nearest neighbor classifier is a non-parametric machine learningmethod in which the input consists of the k closest training examples infeature space. The output is a class membership. An object is classifiedby a plurality vote of its neighbors, with the object being assigned tothe class most common among its k nearest neighbors (k is a positiveinteger, typically small). If k=1, then the object is simply assigned tothe class of that single nearest neighbor. See, Duda et al., 2001,Pattern Classification, Second Edition, John Wiley & Sons, which ishereby incorporated by reference. In some embodiments, the number ofdistance calculations needed to solve the k-nearest neighbor classifieris such that a computer is used to solve the classifier for a giveninput because it cannot be mentally performed.

A deep neural network classifier includes an input layer, a plurality ofindividually weighted convolutional layers, and an output scorer. Theweights of each of the convolutional layers as well as the input layercontribute to the plurality of weights associated with the deep neuralnetwork classifier. In some embodiments, at least 100 weights, at least1000 weights, at least 2000 weights or at least 5000 weights areassociated with the deep neural network classifier. As such, deep neuralnetwork classifiers require a computer to be used because they cannot bementally solved. In other words, given an input to the classifier, theclassifier output needs to be determined using a computer rather thanmentally in such embodiments. See, for example, Krizhevsky et al., 2012,“Imagenet classification with deep convolutional neural networks,” inAdvances in Neural Information Processing Systems 2, Pereira, Burges,Bottou, Weinberger, eds., pp. 1097-1105, Curran Associates, Inc.;Zeiler, 2012 “ADADELTA: an adaptive learning rate method,” CoRR, vol.abs/1212.5701; and Rumelhart et al., 1988, “Neurocomputing: Foundationsof research,” ch. Learning Representations by Back-propagating Errors,pp. 696-699, Cambridge, MA, USA: MIT Press, each of which is herebyincorporated by reference.

SVM classifiers are described in Cristianini and Shawe-Taylor, 2000, “AnIntroduction to Support Vector Machines,” Cambridge University Press,Cambridge; Boser et al., 1992, “A training algorithm for optimal marginclassifiers,” in Proceedings of the 5th Annual ACM Workshop onComputational Learning Theory, ACM Press, Pittsburgh, Pa., pp. 142-152;Vapnik, 1998, Statistical Learning Theory, Wiley, New York; Mount, 2001,Bioinformatics: sequence and genome analysis, Cold Spring HarborLaboratory Press, Cold Spring Harbor, N.Y.; Duda, PatternClassification, Second Edition, 2001, John Wiley & Sons, Inc., pp. 259,262-265; and Hastie, 2001, The Elements of Statistical Learning,Springer, N.Y.; and Furey et al., 2000, Bioinformatics 16, 906-914, eachof which is hereby incorporated by reference in its entirety. When usedfor classification, SVMs separate a given set of binary labeled datatraining set with a hyper-plane that is maximally distant from thelabeled data. For cases in which no linear separation is possible, SVMscan work in combination with the technique of “kernels,” whichautomatically realizes a non-linear mapping to a feature space. Thehyper-plane found by the SVM in feature space corresponds to anon-linear decision boundary in the input space. In some embodiments,the plurality of weights associated with the SVM define the hyper-plane.In some embodiments, the hyper-plane is defined by at least 10, at least20, at least 50, or at least 100 weights and the SVM classifier requiresa computer to calculate because it cannot be mentally solved.

Decision tree classifiers are described generally by Duda, 2001, PatternClassification, John Wiley & Sons, Inc., New York, pp. 395-396, which ishereby incorporated by reference. Tree-based methods partition thefeature space into a set of rectangles, and then fit a model (like aconstant) in each one. In some embodiments, the decision tree is randomforest regression. One specific algorithm that can be used is aclassification and regression tree (CART). Other specific decision treealgorithms include, but are not limited to, ID3, C4.5, MART, and RandomForests. CART, ID3, and C4.5 are described in Duda, 2001, PatternClassification, John Wiley & Sons, Inc., New York, pp. 396-408 and pp.411-412, which is hereby incorporated by reference. CART, MART, and C4.5are described in Hastie et al., 2001, The Elements of StatisticalLearning, Springer-Verlag, New York, Chapter 9, which is herebyincorporated by reference in its entirety. Random Forests are describedin Breiman, 1999, “Random Forests--Random Features,” Technical Report567, Statistics Department, U.C. Berkeley, September 1999, which ishereby incorporated by reference in its entirety. In some embodiments,the decision tree classifier includes at least 10, at least 20, at least50, or at least 100 weights (decisions) and requires a computer tocalculate because it cannot be mentally solved.

A Naive Bayes classifier is any classifier in a family of “probabilisticclassifiers” based on applying Bayes' theorem with strong (naïve)independence assumptions between the features. In some embodiments, theyare coupled with Kernel density estimation. See, for example, Hastie etal., 2001, “The elements of statistical learning: data mining,inference, and prediction,” eds. Tibshirani and Friedman, Springer, NewYork, which is hereby incorporated by reference.

In some embodiments, the model is a trained survival function, alsoknown as a complementary cumulative distribution function. Survivalfunctions relate the time that passes, before some event occurs (e.g.,metastasis), to one or more covariates (e.g., features formed from RNAexpression data, somatic mutation data, clinical data, imaging data,etc.) associated with that quantity of time. Non-limiting examples ofsurvival functions include the Kaplan-Meier estimator, proportionalhazards models such as the Cox proportional hazards model (Cox, David R(1972). “Regression Models and Life-Tables”. Journal of the RoyalStatistical Society, Series B. 34 (2): 187-220. JSTOR 2985181. MR0341758, the content of which is hereby incorporated by reference),poisson regression models, accelerated failure time models,first-hitting-time models, and the like. For more information onsurvival functions, see, for example, Kleinbaum, David G.; Klein,Mitchel (2012), Survival analysis: A Self-learning text (Third ed.),Springer, ISBN 978-1441966452, the content of which is herebyincorporated by reference.

In some embodiments, the one or more models includes at least 2, atleast 3, at least 4, at least 5, at least 6, at least 7, at least 8, atleast 9, at least 10, at least 15, at least 20, at least 25, at least30, at least 35, at least 40, at least 50, at least 60, at least 70, atleast 80, at least 90, or at least 100 models. In some embodiments, theone or more models includes at least 50, at least 100, at least 200, atleast 300, at least 400, at least 500, at least 600, at least 700, atleast 800, at least 900, or at least 1000 models, or more. In someembodiments, the one or more models includes no more than 500, no morethan 400, no more than 300, no more than 200, no more than 100, no morethan 75, no more than 50, no more than 25, or no more than 10 models, orless. In some embodiments, the one or more models includes from 2 to1000 models, from 10 to 500 models, from 10 to 200 models, from 5 to 100models, from 5 to 50 models, from 100 to 200 models, or from 100 to 1000models. In some embodiments the one or more models falls within anotherrange starting no lower than 2 models and ending no higher than 1000models.

In some embodiments, the one or more models is a set of models, eachmodel in the set of models corresponding to a respective differentcombination of a respective tissue in a plurality of tissues and arespective time horizon in a plurality of time horizons for therespective tissue.

For example, in some embodiments, the set of models includes, for eachrespective tissue in the plurality of tissues, a respective subset ofmodels, where each respective model, in the respective subset of models,is trained to provide a respective indication of whether the cancer inthe subject will metastasize to the respective tissue in the subjectwithin a single respective time horizon in the respective plurality oftime horizons (e.g., separate models for each combination of tissue andtime horizon). For example, referring to FIG. 17, in some suchembodiments, the set of models includes N x M models for eachcombination of N tissues and M time horizons, where each model providesa single indication for the nth tissue at the mth time horizon.

In some embodiments, the respective plurality of time horizons for eachrespective tissue in the plurality of tissues is the same, and the setof models includes, for each respective time horizon in the plurality oftime horizons, a respective model trained to provide, for eachrespective tissue in the plurality of tissues, a respective indicationof whether the cancer in the subject will metastasize to the respectivetissue in the subject within the respective time horizon (e.g., separatemulti-label models indicating a plurality of tissues for each timehorizon). For example, referring again to FIG. 17, in some suchembodiments, the set of models includes M models for each of M timehorizons, where each model provides N indications for each of N tissues.

In some embodiments, the set of models includes, for each respectivetissue in the plurality of tissues, a respective model trained toprovide, for each respective time horizon in the plurality of timehorizons, a respective indication of whether the cancer in the subjectwill metastasize to the respective tissue in the subject within therespective time horizon (e.g., separate multi-label models indicating aplurality of time horizons for each tissue). Thus, referring again toFIG. 17, in some such embodiments, the set of models includes N modelsfor each of N tissues, where each model provides M indications for eachof M time horizons.

In some embodiments, the set of models includes a respective modeltrained to provide, for each respective time horizon in a plurality oftime horizons, a respective indication of whether the cancer willmetastasize to any tissue in the subject, and the plurality ofindications of whether the cancer will metastasize includes, for eachrespective time horizon in a plurality of time horizons, a respectiveindication of whether the cancer will metastasize to any tissue in thesubject (e.g., likelihood or metastasis to any tissue in the subject).Thus, in some embodiments, a model in the set of models provides aprediction of cancer metastasis within one or more periods of time, butwithout specifying a predicted metastasis site.

In some embodiments, the one or more models are collectively trained bya process comprising obtaining, in electronic format, for eachrespective training subject in a plurality of training subjects, arespective plurality of data elements. Each respective training subjectin the plurality of training subjects has a respective cancer, and foreach respective training subject in the plurality of training subjects,the respective plurality of data elements comprises a first set ofsequence features comprising relative abundance values for theexpression of a plurality of genes (e.g., at least 30 genes) in a biopsyof the cancer obtained from the training subject. In some embodiments, arespective training subject in the plurality of training subjects,including associated cancer conditions, data elements, sequencefeatures, and/or genes, comprises any of the embodiments for subjects,cancer conditions, data elements, sequence features, and genes describedin the above sections (see, “Cancer conditions” and “Data elements,”above).

In some embodiments, the training the one or more models comprisestraining one or more untrained or partially untrained models using therespective plurality of data elements for each respective trainingsubject in the plurality of training subjects as input, thus obtainingone or more corresponding trained models that are collectively trainedto provide a respective one or more indications of whether a cancer willmetastasize in a subject.

For instance, another aspect of the present disclosure provides a methodfor predicting metastasis of a cancer in a subject, comprising at acomputer system having one or more processors, and memory storing one ormore programs for execution by the one or more processors, obtaining, inelectronic format, for each respective training subject in a pluralityof training subjects, a respective plurality of data elements. Eachrespective training subject in the plurality of training subjects has arespective cancer, and for each respective training subject in theplurality of training subjects, the respective plurality of dataelements comprises a first set of sequence features comprising relativeabundance values for the expression of a plurality of genes (e.g., atleast 30 genes) in a biopsy of the cancer obtained from the trainingsubject. The method comprises training one or more untrained orpartially untrained models using the respective plurality of dataelements for each respective training subject in the plurality oftraining subjects as input, thus obtaining one or more correspondingtrained models, and using the one or more trained models to collectivelyprovide a respective one or more indications of whether a cancer willmetastasize in a test subject, thereby predicting whether the cancerwill metastasize.

In some embodiments, a model in the one or more models is trained andimplemented using any of the methods and/or embodiments for modeltraining and implementation described in the above sections (see,“Artificial Intelligence Engine Training Pipeline,” and “Prediction ofColorectal Metastasis based on RNA”). In some embodiments, the one ormore models is trained using an optional classifier training module 1520comprising any of the training subjects, training data, associatedcancer conditions, data elements, sequence features, and/or genesdisclosed herein.

Suitable methods and/or embodiments for training models, includingfeature selection, identification of prior features, identification offorward features, target and objective selection, and/or model training,include but are not limited to any of the methods and/or embodiments fortraining models, including feature selection, identification of priorfeatures, identification of forward features, target and objectiveselection, and/or model training described in the present disclosure(see, for example, “Generating and Modeling Predictions of PatientObjectives” and FIGS. 6-8), and any substitutions, additions, deletions,modifications, and/or combinations thereof as will be apparent to oneskilled in the art.

In some embodiments, the one or more models, including trained,untrained, and/or partially untrained models, includes any of theembodiments disclosed herein, including any substitutions, additions,deletions, modifications, and/or combinations thereof as will beapparent to one skilled in the art.

Reporting. Referring to Block 1614, in some embodiments, the methodfurther includes generating a clinical report comprising the one or moreindications of whether the cancer will metastasize. In some embodiments,where multiple indications are provided (as described in detail in theforegoing sections), the one or more indications are presented as a listof tissues, a list of time horizons, and/or a list of possiblecombinations of tissues and time horizons, where each entry in therespective list includes a corresponding indication of whether thecancer will metastasize for the respective tissue, time horizon, and/orcombination of tissue and time horizon. In some embodiments, the one ormore indications are presented as a table, a chart, and/or a graphicalrepresentation. In some embodiments, the one or more indications arepresented in an interactive format.

In some embodiments, the method further includes displaying the clinicalreport in a graphical user interface (GUI), wherein the GUI includes ananatomical representation of a body and a first affordance configuredfor switching between respective time horizons in the plurality of timehorizons. The displaying includes displaying a first rendering ofmetastatic predictions comprising, for each respective tissue in theplurality of tissues, a respective visual representation of therespective indication, in the plurality of indications, corresponding towhether the cancer in the subject will metastasize to the respectivetissue within a first respective time horizon in the plurality of timehorizons, where the rendering is superposed upon the anatomicalrepresentation of the body. Responsive to receiving a user inputcorresponding to the first affordance on the GUI, the method includesreplacing the display of the first rendering of metastatic predictionswith a display of a second rendering of metastatic predictionscomprising, for each respective tissue in the plurality of tissues, arespective visual representation of the respective indication, in theplurality of indications, corresponding to whether the cancer in thesubject will metastasize to the respective tissue within a secondrespective time horizon in the plurality of time horizons, where therendering is superposed upon the anatomical representation of the body.

In some embodiments, the graphical user interface is a prediction toolsuch as a webform, as described in detail in the present disclosure andwith reference to FIGS. 9-12 (see, “Generating and Modeling Predictionsof Patient Objectives”).

Referring to Block 1616, in some embodiments, the method furtherincludes, when the one or more indications of whether the cancer willmetastasize satisfy a first threshold risk for metastasis of the cancer,administering a first therapy tailored for treatment of metastaticcancer, and when the one or more indications of whether the cancer willmetastasize do not satisfy the first threshold risk for metastasis ofthe cancer, administering a second therapy tailored for treatment ofnon-metastatic cancer. For example, in some embodiments, the firstthreshold risk for metastasis is an indication that the cancer of thesubject will metastasize (e.g., has above a threshold probability ofmetastasizing) to any one or more tissues within a first time horizon(e.g., within 6 months, within 1 year, within 2 years, within 3 years,within 4 years, within 5 years, within 10 years, and/or within greaterthan 10 years). In some embodiments, the first threshold risk formetastasis is an indication that the cancer of the subject willmetastasize (e.g., has above a threshold probability of metastasizing)to a respective tissue (e.g., a specific metastasis site), in aplurality of tissues, within a first time horizon.

In some embodiments, the subject is administered an anti-cancer agentselected from lenalidomid, pembrolizumab, trastuzumab, bevacizumab,rituximab, ibrutinib, human papillomavirus quadrivalent (types 6, 11,16, and 18) vaccine, pertuzumab, pemetrexed, nilotinib, nilotinib,denosumab, abiraterone acetate, promacta, imatinib, everolimus,palbociclib, erlotinib, bortezomib, and bortezomib. In some embodiments,the first therapy is an anti-cancer agent that is selected based on theidentity of one or more tissues, in the plurality of tissues, predictedfor metastasis. In some embodiments, the first therapy and/or the secondtherapy is administered in accordance with the NCCN standard of careguidelines (available online at www.nccn.org). In some embodiments, thesubject is administered with a combination therapy. For example, in someembodiments, the first therapy and/or the second therapy is administeredin conjunction with a radiation therapy and/or a surgical treatment.

Additional Embodiments

Another aspect of the present disclosure provides a method forpredicting metastasis of a cancer in a subject, comprising obtaining, inelectronic format, a plurality of data elements for the subject'scancer. In some embodiments, the cancer is colorectal cancer, non-smallcell lung cancer (NSCLC), breast cancer, or ovarian cancer. In someembodiments, the cancer is colon cancer. In some embodiments, the canceris rectosigmoid junction cancer. In some embodiments, the cancer iscolon or rectosigmoid junction cancer.

In some embodiments, the plurality of data elements includes a first setof sequence features based on relative abundance values for theexpression of a plurality of genes (e.g., at least 30 genes) in a biopsyof the cancer obtained from the subject.

In some embodiments, the plurality of data elements also includes one ormore personal characteristics about the subject (e.g., selected from thegroup consisting of age, gender, and race).

In some embodiments, the plurality of data elements further includes oneor more clinical features related to the diagnosis or treatment of thecancer in the subject selected from the group consisting of a stage ofthe cancer, a histopathological grade of the cancer, a therapyadministered to the subject, a symptom associated with cancer ormetastasis thereof, and a comorbidity with the cancer.

In some embodiments, the plurality of data elements further includes oneor more temporal features (e.g., associated with any of the clinicalfeatures in the one or more clinical features) related to the diagnosisor treatment of the cancer in the subject selected from the groupconsisting of a first temporal element indicating the duration of timesince a diagnosis for the cancer, a second temporal element indicatingthe duration of time since an administration of the therapy, a thirdtemporal element indicating the duration of time since an experience ofthe symptom, and a fourth temporal element indicating the duration oftime since an experience of the comorbidity.

Suitable methods and/or embodiments for obtaining the plurality of dataelements, including sequence features (e.g., collecting biopsies,sequencing methods, sequence reads, preprocessing methods, determiningabundance values, and/or selection of sequence features), personalcharacteristics, clinical features, and/or additional data elements,include but are not limited to any of the methods and/or embodiments forobtaining a plurality of data elements described in the presentdisclosure, including sequence features (e.g., collecting biopsies,sequencing methods, sequence reads, preprocessing methods, determiningabundance values, and/or selection of sequence features), personalcharacteristics, clinical features, and/or additional data elements(see, “Classification Methods: Data elements”), and any substitutions,additions, deletions, modifications, and/or combinations thereof as willbe apparent to one skilled in the art.

For example, in some embodiments, obtaining the first set of sequencefeatures includes obtaining a plurality of at least 10,000 sequencereads, where the plurality of sequence reads is obtained from aplurality of RNA molecules from the biopsy of the cancer obtained fromthe subject, and determining, from the plurality of sequence reads,relative abundance values for a plurality of genes. In some embodiments,the plurality of genes comprises at least 20 genes selected from thegroup consisting of the genes listed in Table 2. In some embodiments,the plurality of genes is no more than 250 genes.

In some embodiments, the first set of sequence features includesrelative abundance values for the expression of a plurality of genes. Insome embodiments, the first set of features includes relative abundancevalues for the expression of at least 25 genes. In some embodiments, thefirst set of features includes relative abundance values for theexpression of at least 50 genes. In some embodiments, the first set offeatures includes relative abundance values for the expression of atleast 75 genes. In some embodiments, the first set of features includesrelative abundance values for the expression of at least 100 genes. Insome embodiments, the first set of features includes relative abundancevalues for the expression of at least 250 genes. In some embodiments,the first set of features includes relative abundance values for theexpression of at least 150 genes, at least 200 genes, at least 300genes, at least 400 genes, at least 500 genes, at least 750 genes, atleast 1000 genes, or more.

In some embodiments, the first set of sequence features includes aplurality of dimension reduction component values determined fromrelative abundance values for a plurality of genes. For instance,Example 1 describes an instance where relative expression values for the500 most correlated genes are used to generate 40 sequence featuresusing singular value decomposition, which are the basis for a predictionmodel. Accordingly, in some embodiments, the first set of sequencefeatures include a plurality of dimension reduction component valuesdetermined from relative abundance values for at least 25, at least 50,at least 75, at least 100, at least 150, at least 200, at least 250, atleast 500, at least 750, at least 1000, or more genes. In someembodiments, the first set of sequence features include a plurality ofdimension reduction component values determined from relative abundancevalues for no more than 2000 genes, no more than 1500 genes, no morethan 1000 genes, no more than 750 genes, no more than 500 genes, orless. In some embodiments, the first set of sequence features include aplurality of dimension reduction component values determined fromrelative abundance values for from 25 to 2000 genes, from 50 to 1000genes, or from 100 to 750 genes.

In some embodiments, the plurality of data elements further includes asingle-sample gene set enrichment analysis (ssGSEA) score for thetranscriptional profile of the cancer.

In some embodiments, the plurality of data elements further includes amutational status for one or more genes in the genome of the cancer(e.g., where the mutational status is for a gene selected from the groupconsisting of the genes listed in Table 2). In some embodiments, theplurality of data elements further includes a mutational status for oneor more genes in the genome of a non-cancerous tissue of the subject. Insome embodiments, the plurality of data elements further includes a copynumber status for one or more genomic regions (e.g., one or more genes)of the cancer.

In some embodiments, the plurality of data elements further includes apersonal characteristic. In some embodiments, the one or more personalcharacteristics include smoking status or menopausal status (e.g.,associated with lung cancer, breast cancer, and/or ovarian cancer). Insome embodiments, the plurality of data elements further includes aclinical feature for the subject.

In some embodiments, the plurality of data elements further includes aphysical characteristic of the biopsy of the cancer. For example, insome embodiments, the plurality of data elements includes tumorpercentage, tumor cell percentage, tumor infiltrating lymphocytespercentage, tumor budding features, biologically meaningful features,and/or geometrically meaningful features (e.g., aggregation metrics(e.g., minimum, average, median, maximum) of tumor perimeter, averagetumor cell circularity, average tumor cell length and/or aspect ratio).

The method further includes applying, to the plurality of data elementsfor the subject's cancer, a model (e.g., a predictive and/orclassification model) that is collectively trained to provide anindication (e.g., binary, likelihood, and/or probability) of whether thecancer will metastasize in the subject, thus predicting whether thecancer will metastasize. Thus, in some embodiments, the presentdisclosure provides a single model that provides a single indication ofwhether the cancer will metastasize in the subject to any tissue site inthe subject within a particular time horizon.

Suitable methods and/or embodiments for predicting whether the cancerwill metastasize using a model, including tissues, time horizons,indications, types of models, training models, and/or outputs, includebut are not limited to any of the methods and/or embodiments forobtaining indications described in the present disclosure, includingtissues, time horizons, indications, types of models, training models,and/or outputs (see, “Classification Methods: Obtaining indications ofmetastasis”), and any substitutions, additions, deletions,modifications, and/or combinations thereof as will be apparent to oneskilled in the art.

In some embodiments, the method comprises applying, to the plurality ofdata elements for the subject's cancer, a set of models that arecollectively trained to provide, for each respective tissue in aplurality of tissues, a respective set of indications in the one or moreindications of whether the cancer will metastasize to the respectivetissue in the subject, where the respective set of indications includesa respective indication for each respective time horizon in a pluralityof time horizons, thus determining a plurality of indications of whetherthe cancer will metastasize comprising, for each respective tissue inthe plurality of tissues, a respective set of indications comprising,for each respective time horizon in a plurality of time horizons, arespective indication of whether the cancer in the subject willmetastasize to the respective tissue within the respective time horizon.

In some embodiments, the set of models comprises, for each respectivetissue in the plurality of tissues, a respective subset of models, whereeach respective model, in the respective subset of models, is trained toprovide a respective indication of whether the cancer in the subjectwill metastasize to the respective tissue in the subject within a singlerespective time horizon in the respective plurality of time horizons.

In some embodiments, the respective plurality of time horizons for eachrespective tissue in the plurality of tissues is the same, and the setof models comprises, for each respective time horizon in the pluralityof time horizons, a respective model trained to provide, for eachrespective tissue in the plurality of tissues, a respective indicationof whether the cancer in the subject will metastasize to the respectivetissue in the subject within the respective time horizon.

In some embodiments, the set of models comprises, for each respectivetissue in the plurality of tissues, a respective model trained toprovide, for each respective time horizon in the plurality of timehorizons, a respective indication of whether the cancer in the subjectwill metastasize to the respective tissue in the subject within therespective time horizon.

In some embodiments, the set of models comprises a respective modeltrained to provide, for each respective time horizon in a plurality oftime horizons, a respective indication of whether the cancer willmetastasize to any tissue in the subject, and the plurality ofindications of whether the cancer will metastasize includes, for eachrespective time horizon in a plurality of time horizons, a respectiveindication of whether the cancer will metastasize to any tissue in thesubject.

In some embodiments, the plurality of tissues comprises lymph tissue,liver tissue, and lung tissue.

In some embodiments, the one or more trained models are collectivelytrained by a process comprising obtaining, in electronic format, foreach respective training subject in a plurality of training subjects, arespective plurality of data elements. Each respective training subjectin the plurality of training subjects has a respective cancer, and foreach respective training subject in the plurality of training subjects,the respective plurality of data elements comprises a first set ofsequence features comprising relative abundance values for theexpression of a plurality of genes (e.g., at least 30 genes) in a biopsyof the cancer obtained from the training subject. The method includestraining one or more untrained or partially untrained models using therespective plurality of data elements for each respective trainingsubject in the plurality of training subjects as input, thus obtainingone or more corresponding trained models that are collectively trainedto provide a respective one or more indications of whether a cancer willmetastasize in a subject.

Suitable methods and/or embodiments for training models, includingfeature selection, identification of prior features, identification offorward features, target and objective selection, and/or model training,include but are not limited to any of the methods and/or embodiments fortraining models, including feature selection, identification of priorfeatures, identification of forward features, target and objectiveselection, and/or model training described in the present disclosure(see, for example, “Generating and Modeling Predictions of PatientObjectives,” with reference to FIGS. 6-8, and “Classification Methods:Obtaining indications of metastasis”), and any substitutions, additions,deletions, modifications, and/or combinations thereof as will beapparent to one skilled in the art.

The method further includes generating a clinical report comprising theindication of whether the cancer will metastasize.

In some embodiments, the method further includes displaying the clinicalreport in a graphical user interface (GUI), where the GUI comprises ananatomical representation of a body and a first affordance configuredfor switching between respective time horizons in the plurality of timehorizons. The displaying comprises displaying a first rendering ofmetastatic predictions comprising, for each respective tissue in theplurality of tissues, a respective visual representation of therespective indication, in the plurality of indications, corresponding towhether the cancer in the subject will metastasize to the respectivetissue within a first respective time horizon in the plurality of timehorizons, where the rendering is superposed upon the anatomicalrepresentation of the body. Responsive to receiving a user inputcorresponding to the first affordance on the GUI, the method includesreplacing display of the first rendering of metastatic predictions withdisplay of a second rendering of metastatic predictions comprising, foreach respective tissue in the plurality of tissues, a respective visualrepresentation of the respective indication, in the plurality ofindications, corresponding to whether the cancer in the subject willmetastasize to the respective tissue within a second respective timehorizon in the plurality of time horizons, where the rendering issuperposed upon the anatomical representation of the body.

In some embodiments, the method further includes, when the indication ofwhether the cancer will metastasize satisfies a first threshold risk formetastasis of the cancer, administering a first therapy tailored fortreatment of metastatic cancer, and when the indication of whether thecancer will metastasize does not satisfy the first threshold risk formetastasis of the cancer, administering a second therapy tailored fortreatment of non-metastatic cancer.

In some embodiments, the generating a clinical report and/oradministering therapies using the single model method includes any ofthe methods and/or embodiments for a set of models described in thepresent disclosure, above (see, “Classification Methods: Obtainingindications of metastasis: Reporting”), and any substitutions,additions, deletions, modifications, and/or combinations thereof as willbe apparent to one skilled in the art.

In some embodiments, any of the presently disclosed methods and/orembodiments are performed at a computer system having one or moreprocessors, and memory storing one or more programs for execution by theone or more processors.

Another aspect of the present disclosure provides a computer systemhaving one or more processors, and memory storing one or more programsfor execution by the one or more processors, the one or more programscomprising instructions for performing any of the methods disclosedherein.

Another aspect of the present disclosure provides a non-transitorycomputer readable storage medium storing one or more programs configuredfor execution by a computer, the one or more programs comprisinginstructions for carrying out any of the methods disclosed herein.

Example Models for Predicting Whether a Cancer will Metastasize.

Advantageously, the present disclosure describes several classes offeatures that are informative for predicting whether a cancer willmetastasize. For instance, among other aspects, the present disclosuredescribes improvements in generating predictions of whether a cancerwill metastasize using (i) sequence features comprising relativeabundance values for the expression of a plurality of genes (e.g., atleast 30 genes) in a biopsy of the cancer obtained from the subject,(ii) one or more personal characteristics about the subject, (iii) oneor more clinical features related to the diagnosis or treatment of thecancer in the subject, (iv) one or more temporal features related to thediagnosis or treatment of the cancer in the subject and/or (v) one ormore pathological features of the cancer tissue. It is contemplated thatvarious combinations of these improvements, as well as othernon-conventional aspects described herein, may be integrated into acommon classification module 1536 (e.g., a prediction analysis pipeline)for indicating whether a cancer will metastasize. For instance, in someembodiments, a classification module integrates one, two, three, four,or all five of these feature types for improved predictions of cancermetastasis. Examples of various combinations of improvements that may becombined into a single classification module (e.g., comprising one ormore models), methods associated thereof, systems for performing suchmethods, and/or non-transitory computer readable media for executingsuch methods are described below. It will be appreciated that thesecombinations can be performed with any other preparatory orclassification steps described in the other methods described herein,e.g., methods 600, 800, and 1602 as illustrated in FIGS. 6, 8, and 16,and further described above.

In some embodiments, a classification module for predicting metastasisof a cancer in a subject is provided that integrates at least a firstset of sequence features comprising relative abundance values for theexpression of a plurality of genes, as described above in the sectionentitled “Sequence features.”

Accordingly, in some embodiments, a method is provided for predictingmetastasis of a cancer in a subject that includes (A) obtaining, inelectronic format, a plurality of data elements for the subject's cancercomprising a first set of sequence features comprising relativeabundance values for the expression of a plurality of genes (e.g., atleast 30 genes) in a biopsy of the cancer obtained from the subject, asdescribed above in the section entitled “Sequence features,” (B)applying, to the plurality of data elements for the subject's cancer,one or more models that are collectively trained to provide a respectiveone or more indications of whether the cancer will metastasize in thesubject, thereby predicting whether the cancer will metastasize, and (C)generating a clinical report comprising the one or more indications ofwhether the cancer will metastasize. In some embodiments, the methodincludes, when the one or more indications of whether the cancer willmetastasize satisfies a first threshold risk for metastasis of thecancer, administering a first therapy tailored for treatment ofmetastatic cancer, and when the one or more indications of whether thecancer will metastasize does not satisfy the first threshold risk formetastasis of the cancer, administering a second therapy tailored fortreatment of non-metastatic cancer, as described in the above sectionentitled “Reporting.”

In some embodiments, the plurality of data elements including therelative abundance values for the expression of the plurality of genesalso includes one or more personal characteristics about the subject,e.g., as described above in the section entitled “Personalcharacteristics.” In some embodiments, the plurality of data elementsincluding the relative abundance values for the expression of theplurality of genes also includes the age of the subject. In someembodiments, the plurality of data elements including the relativeabundance values for the expression of the plurality of genes alsoincludes the gender of the subject. In some embodiments, the pluralityof data elements including the relative abundance values for theexpression of the plurality of genes also includes a race of thesubject. In some embodiments, the plurality of data elements includingthe relative abundance values for the expression of the plurality ofgenes also includes a habit of the subject (e.g., a smoking status,alcohol consumption status, dietary status, etc.). In some embodiments,the plurality of data elements including the relative abundance valuesfor the expression of the plurality of genes also includes aphysiological characteristic of the subject (e.g., a blood pressurestatus, a dermatological condition, a co-morbidity, etc.).

In some embodiments, the classification module integrating at least afirst set of sequence features comprising relative abundance values forthe expression of a plurality of genes also integrates one or moreclinical features related to the diagnosis or treatment of the cancer inthe subject, as described above in the section entitled “Clinicalfeatures.” In some embodiments, the classification module integrating atleast a first set of sequence features comprising relative abundancevalues for the expression of a plurality of genes also integrates one ormore temporal features related to the diagnosis or treatment of thecancer in the subject, as described above in the section entitled“Temporal features.”

In some embodiments, the classification module integrating at least afirst set of sequence features comprising relative abundance values forthe expression of a plurality of genes also integrates one or morepersonal characteristics about the subject, as described above in thesection entitled “Personal characteristics,” and further integrates oneor more clinical features related to the diagnosis or treatment of thecancer in the subject, as described above in the section entitled“Clinical features.” In some embodiments, the classification moduleintegrating at least a first set of sequence features comprisingrelative abundance values for the expression of a plurality of genesalso integrates one or more personal characteristics about the subject,as described above in the section entitled “Personal characteristics,”and further integrates one or more temporal features related to thediagnosis or treatment of the cancer in the subject, as described abovein the section entitled “Temporal features.” In some embodiments, theclassification module integrating at least a first set of sequencefeatures comprising relative abundance values for the expression of aplurality of genes also integrates one or more clinical features relatedto the diagnosis or treatment of the cancer in the subject, as describedabove in the section entitled “Clinical features,” and furtherintegrates one or more temporal features related to the diagnosis ortreatment of the cancer in the subject, as described above in thesection entitled “Temporal features.”

In some embodiments, the classification module integrating at least afirst set of sequence features comprising relative abundance values forthe expression of a plurality of genes also integrates one or morepersonal characteristics about the subject, as described above in thesection entitled “Personal characteristics,” further integrates one ormore clinical features related to the diagnosis or treatment of thecancer in the subject, as described above in the section entitled“Clinical features,” and further integrates one or more temporalfeatures related to the diagnosis or treatment of the cancer in thesubject, as described above in the section entitled “Temporal features.”

In some embodiments, a classification module for predicting metastasisof a cancer in a subject is provided that integrates at least a firstset of sequence features comprising relative abundance values for theexpression of a plurality of genes, as described above in the sectionentitled “Sequence features,” and one or more personal characteristicsabout the subject, as described above in the section entitled “Personalcharacteristics.”

Accordingly, in some embodiments, a method is provided for predictingmetastasis of a cancer in a subject that includes (A) obtaining, inelectronic format, a plurality of data elements for the subject's cancercomprising (i) a first set of sequence features comprising relativeabundance values for the expression of a plurality of genes (e.g., atleast 30 genes) in a biopsy of the cancer obtained from the subject, asdescribed above in the section entitled “Sequence features,” and (ii)one or more personal characteristics about the subject selected from thegroup consisting of age, gender, and race, as described above in thesection entitled “Personal characteristics,” (B) applying, to theplurality of data elements for the subject's cancer, one or more modelsthat are collectively trained to provide a respective one or moreindications of whether the cancer will metastasize in the subject,thereby predicting whether the cancer will metastasize, and (C)generating a clinical report comprising the one or more indications ofwhether the cancer will metastasize. In some embodiments, the methodincludes, when the one or more indications of whether the cancer willmetastasize satisfies a first threshold risk for metastasis of thecancer, administering a first therapy tailored for treatment ofmetastatic cancer, and when the one or more indications of whether thecancer will metastasize does not satisfy the first threshold risk formetastasis of the cancer, administering a second therapy tailored fortreatment of non-metastatic cancer, as described in the above sectionentitled “Reporting.”

In some embodiments, the classification module integrating at least afirst set of sequence features comprising relative abundance values forthe expression of a plurality of genes and one or more personalcharacteristics about the subject, also integrates one or more clinicalfeatures related to the diagnosis or treatment of the cancer in thesubject, as described above in the section entitled “Clinical features.”In some embodiments, the classification module integrating at least afirst set of sequence features comprising relative abundance values forthe expression of a plurality of genes and one or more personalcharacteristics about the subject, also integrates one or more temporalfeatures related to the diagnosis or treatment of the cancer in thesubject, as described above in the section entitled “Temporal features.”

In some embodiments, the classification module integrating at least afirst set of sequence features comprising relative abundance values forthe expression of a plurality of genes and one or more personalcharacteristics about the subject, also integrates one or more clinicalfeatures related to the diagnosis or treatment of the cancer in thesubject, as described above in the section entitled “Clinical features,”and further integrates one or more temporal features related to thediagnosis or treatment of the cancer in the subject, as described abovein the section entitled “Temporal features.”

In some embodiments, a classification module for predicting metastasisof a cancer in a subject is provided that integrates at least a firstset of sequence features comprising relative abundance values for theexpression of a plurality of genes, as described above in the sectionentitled “Sequence features,” and one or more clinical features relatedto the diagnosis or treatment of the cancer in the subject, as describedabove in the section entitled “Clinical features.”

Accordingly, in some embodiments, a method is provided for predictingmetastasis of a cancer in a subject that includes (A) obtaining, inelectronic format, a plurality of data elements for the subject's cancercomprising (i) a first set of sequence features comprising relativeabundance values for the expression of a plurality of genes (e.g., atleast 30 genes) in a biopsy of the cancer obtained from the subject, asdescribed above in the section entitled “Sequence features,” and (ii)one or more clinical features related to the diagnosis or treatment ofthe cancer in the subject selected from the group consisting of a stageof the cancer, a histopathological grade of the cancer, a therapyadministered to the subject, a symptom associated with cancer ormetastasis thereof, and a comorbidity with the cancer, as describedabove in the section entitled “Clinical features,” (B) applying, to theplurality of data elements for the subject's cancer, one or more modelsthat are collectively trained to provide a respective one or moreindications of whether the cancer will metastasize in the subject,thereby predicting whether the cancer will metastasize, and (C)generating a clinical report comprising the one or more indications ofwhether the cancer will metastasize. In some embodiments, the methodincludes, when the one or more indications of whether the cancer willmetastasize satisfies a first threshold risk for metastasis of thecancer, administering a first therapy tailored for treatment ofmetastatic cancer, and when the one or more indications of whether thecancer will metastasize does not satisfy the first threshold risk formetastasis of the cancer, administering a second therapy tailored fortreatment of non-metastatic cancer, as described in the above sectionentitled “Reporting.”

In some embodiments, the classification module integrating at least afirst set of sequence features comprising relative abundance values forthe expression of a plurality of genes and one or more clinical featuresrelated to the diagnosis or treatment of the cancer in the subject, alsointegrates one or more personal characteristics about the subject, asdescribed above in the section entitled “Personal characteristics.” Insome embodiments, the classification module integrating at least a firstset of sequence features comprising relative abundance values for theexpression of a plurality of genes and one or more clinical featuresrelated to the diagnosis or treatment of the cancer in the subject, alsointegrates one or more temporal features related to the diagnosis ortreatment of the cancer in the subject, as described above in thesection entitled “Temporal features.”

In some embodiments, the classification module integrating at least afirst set of sequence features comprising relative abundance values forthe expression of a plurality of genes and one or more clinical featuresrelated to the diagnosis or treatment of the cancer in the subject, alsointegrates one or more personal characteristics about the subject, asdescribed above in the section entitled “Personal characteristics,” andfurther integrates one or more temporal features related to thediagnosis or treatment of the cancer in the subject, as described abovein the section entitled “Temporal features.”

In some embodiments, a classification module for predicting metastasisof a cancer in a subject is provided that integrates at least a firstset of sequence features comprising relative abundance values for theexpression of a plurality of genes, as described above in the sectionentitled “Sequence features,” and one or more temporal features relatedto the diagnosis or treatment of the cancer in the subject, as describedabove in the section entitled “Temporal features.”

Accordingly, in some embodiments, a method is provided for predictingmetastasis of a cancer in a subject that includes (A) obtaining, inelectronic format, a plurality of data elements for the subject's cancercomprising (i) a first set of sequence features comprising relativeabundance values for the expression of a plurality of genes (e.g., atleast 30 genes) in a biopsy of the cancer obtained from the subject, asdescribed above in the section entitled “Sequence features,” and (ii)one or more temporal features related to the diagnosis or treatment ofthe cancer in the subject selected from the group consisting of a firsttemporal element indicating a duration of time since a diagnosis for thecancer, a second temporal element indicating a duration of time since anadministration of a therapy to the subject, a third temporal elementindicating a duration of time since an experience of a symptomassociated with cancer or metastasis thereof, and a fourth temporalelement indicating a duration of time since an experience of acomorbidity with the cancer, as described above in the section entitled“Temporal features,” (B) applying, to the plurality of data elements forthe subject's cancer, one or more models that are collectively trainedto provide a respective one or more indications of whether the cancerwill metastasize in the subject, thereby predicting whether the cancerwill metastasize, and (C) generating a clinical report comprising theone or more indications of whether the cancer will metastasize. In someembodiments, the method includes, when the one or more indications ofwhether the cancer will metastasize satisfies a first threshold risk formetastasis of the cancer, administering a first therapy tailored fortreatment of metastatic cancer, and when the one or more indications ofwhether the cancer will metastasize does not satisfy the first thresholdrisk for metastasis of the cancer, administering a second therapytailored for treatment of non-metastatic cancer, as described in theabove section entitled “Reporting.”

In some embodiments, the classification module integrating at least afirst set of sequence features comprising relative abundance values forthe expression of a plurality of genes and one or more temporal featuresrelated to the diagnosis or treatment of the cancer in the subject, alsointegrates one or more personal characteristics about the subject, asdescribed above in the section entitled “Personal characteristics.” Insome embodiments, the classification module integrating at least a firstset of sequence features comprising relative abundance values for theexpression of a plurality of genes and one or more temporal featuresrelated to the diagnosis or treatment of the cancer in the subject, alsointegrates one or more clinical features related to the diagnosis ortreatment of the cancer in the subject, as described above in thesection entitled “Clinical features.”

In some embodiments, the classification module integrating at least afirst set of sequence features comprising relative abundance values forthe expression of a plurality of genes and one or more temporal featuresrelated to the diagnosis or treatment of the cancer in the subject, alsointegrates one or more personal characteristics about the subject, asdescribed above in the section entitled “Personal characteristics,” andfurther integrates one or more clinical features related to thediagnosis or treatment of the cancer in the subject, as described abovein the section entitled “Clinical features.”

Specific Embodiments

The following clauses describe specific embodiments of the disclosure.

Clause 1. A method, comprising receiving patient information comprisingresult of analysis of a first plurality of nucleic acids derived from asomatic specimen and a second plurality of nucleic acids derived from agermline specimen of a plurality of patients; identifying one or moreinteractions for each of the plurality of patients based at least inpart on the received patient information; generating, for one or moretargets associated with each interaction of the one or moreinteractions, one or more timeline metrics identifying whether each ofthe one or more targets occurs within a time period of an occurrence ofthe interaction; determining, for each timeline metric of the one ormore timeline metrics, a probability that a patient will be associatedwith one or more status characteristics within the time period; traininga target prediction model for each of the one or more targets based atleast in part on the one or more status characteristics therebygenerating a plurality of trained target prediction models; andassociating predictions for each patient, wherein the predictions aregenerated by the target prediction model for each of the one or moretargets, with a respective timeline metric from the one or more timelinemetrics.

Clause 2. The method of clause 1, further comprising rendering, on agraphical user interface of a computing device, a representation of thepredictions in association with the respective timeline metric.

Clause 3. The method of clause 1, further comprising receiving secondinformation associated with a new patient; identifying at least oneinteraction from the second information; selecting a target predictionmodel from the plurality of trained target prediction models based on atype of the received second information; and applying the selectedtarget prediction model to the second information to generatepredictions for each target from the one or more targets, wherein eachtarget corresponds to an interaction from the at least one interaction,and wherein each target is associated with a timeline metric from theone or more timeline metrics.

Clause 4. The method of clause 3, further comprising rendering, on agraphical user interface of a computing device, a representation of thepredictions for each target in association with the respective timelinemetric.

Clause 5. The method of any one of clauses 1 to 4, wherein the patientinformation includes information acquired from an electronic medicalrecord (EMR) and/or free-text progress notes for each patient.

Clause 6. The method of any one of clauses 1 to 5, wherein the patientinformation includes one or more of clinical information, informationobtained using immunohistochemistry (IHC), and information obtainedusing fluorescence in situ hybridization (FISH).

Clause 7. The method of any one of clauses 1 to 5, wherein the patientinformation includes one or more of clinical information, informationobtained from pathology reports, and information obtained from radiologyreports.

Clause 8. The method of any one of clauses 1 to 7, wherein the one ormore targets comprise one or more of metastasis to an organ of thepatient, a measure of cancer progression in the patient, cancer localrecurrence in the patient, and cancer regional recurrence in thepatient.

Clause 9. The method of any one of clauses 1 to 8, wherein the organ isthe brain, lung, breast, liver, pancreas, colon, skin, lymph nodes, andbones.

Clause 10. The method of any one of clauses 1 to 9, wherein the timeperiod is measured in days, month, or years.

Clause 11. The method of any one of clauses 1 to 10, wherein the one ormore interactions comprise a record from a patient's medical history, arecord of a diagnosis, a record of a prescribed medication, a record ofa taken medication, a record of an administered treatment, a record of acancer progression, a record of a cancer recurrence, a record of acancer localized metastasis, a record of a genetic sequencing, or arecord of a digital image acquisition.

Clause 12. The method of clause 11, wherein the record of theadministered treatment comprises a record of one or more of surgery,therapy, or procedure.

Clause 13. The method of clause 11, wherein the record of the digitalimage acquisition is information obtained from an H&E slide, IHC slide,or a radiology image.

Clause 14. The method of any one of clauses 1 to 13, wherein the one ormore status characteristics comprise a prior occurrence of aninteraction or a prior result of a laboratory test.

Clause 15. The method of any one of clauses 1 to 14, wherein the one ormore status characteristics are measured as a time since occurrence ofone or more prior interaction.

Clause 16. The method of any one of clauses 1 to 15, wherein the patientis a cancer patient.

Clause 17. The method of clause 16, wherein the cancer comprises adrenalcancer, biliary tract cancer, bladder cancer, bone/bone marrow cancer,brain cancer, breast cancer, cervical cancer, colorectal cancer, cancerof the esophagus, gastric cancer, head/neck cancer, hepatobiliarycancer, kidney cancer, liver cancer, lung cancer, ovarian cancer,pancreatic cancer, pelvis cancer, pleura cancer, prostate cancer, renalcancer, skin cancer, stomach cancer, testis cancer, thymus cancer,thyroid cancer, uterine cancer, lymphoma, melanoma, multiple myeloma,leukemia, or a combination thereof.

Clause 18. The method of any one of clauses 1 to 17, further comprisingautomatically generating an electronic report including the predictionsgenerated for the one or more targets, and respective timeline metrics.

Clause 19. The method of clause 18, comprising transmitting theelectronic report to a user over a computer network in real time, sothat the user has immediate access to the electronic report.

Clause 20. The method of any one of clauses 1 to 17, further comprisingreceiving an anchor point; identifying one or more subsets of patientsfrom the plurality of patients for which targets associated withrespective predictions, at the anchor point, deviate from observedtarget occurrence for a control cohort of patients; and automaticallygenerating an electronic report indicating a likelihood of a patient,from the one or more subsets of patients, experiencing metastasis to oneor more organs.

Clause 21. The method of clause 20, wherein the anchor point is anoccurrence of next generation genetic sequencing of the patient's tumor.

Clause 22. The method of clause 21, wherein the next generation geneticsequencing of the patient's tumor includes tumor-normal matchedsequencing, full transcriptome sequencing, tumor-only sequencing, orcell-free DNA sequencing of the patient's blood.

Clause 23. The method of clause 18 or 20, wherein the electronic reportis generated as part of a precision medicine result delivery for thepatient.

Clause 24. The method of clause 18 or 20, wherein the electronic reportcomprises a recommendation to a physician to treat the patient using atreatment that correlates with a magnitude of a determined degree ofrisk of the metastasis.

Clause 25. The method of clause 18 or 20, wherein the electronic reportcomprises a recommendation to a physician to select a treatment whichprovides adjustments to a typical monitoring including one or more ofscanning, imaging, and blood testing.

Clause 26. The method of clause 18 or 20, wherein the electronic reportcomprises a recommendation of accelerated screening.

Clause 27. The method of any one of clauses 1 to 26, wherein the methodoperates as part of a digital and laboratory health care platform.

Clause 28. The method of clause 27, wherein the digital and laboratoryhealth care platform generates a molecular report as part of a targetedmedical care precision medicine treatment.

Clause 29. The method of any one of clauses 1 to 26, wherein the methodoperates on one or more micro-services.

Clause 30. The method of any one of clauses 1 to 26, wherein the methodis performed in conjunction with one or more micro-services of an ordermanagement system.

Clause 31. The method of any one of clauses 1 to 26, wherein the methodis performed in conjunction with one or more micro-services of acell-type profiling service.

Clause 32. The method of any one of clauses 1 to 26, wherein the methodis performed in conjunction with a variant calling engine and/or aninsight engine.

Clause 33. The method of any one of clauses 1 to 32, wherein the one ormore status characteristics comprise a plurality of features.

Clause 34. The method of clause 33, comprising applying a dimensionalityreduction algorithm to the plurality of features to generate a reducedset of a plurality of features.

Clause 35. The method of any one of clauses 1 to 32, wherein the patientinformation comprises a plurality of patient features.

Clause 36. The method of clause 33, comprising applying a dimensionalityreduction algorithm to the plurality of patient features to generate areduced set of a plurality of features.

Clause 37. The method of clause 34 or 36, wherein the dimensionalityreduction algorithm comprises a supervised algorithm.

Clause 38. The method of clause 37, wherein the supervised algorithmcomprises one or more of Linear Discriminant Analysis, NeighborhoodComponent Analysis, MLP transfer learning, and tree-based supervisedembedding.

Clause 39. The method of clause 34 or 36, wherein the dimensionalityreduction algorithm comprises an unsupervised algorithm.

Clause 40. The method of clause 39, wherein the unsupervised algorithmcomprises one or more of an RNA Variational Auto-encoder, Singular ValueDecomposition (SVD), PCA, KernelPCA, SparsePCA, DictionaryLearning,Isomap, Nonnegative Matrix Factorization (NMF), Uniform ManifoldApproximation and Projection (UMAP), feature agglomeration, patientcorrelation clustering, KMeans, Gaussian Mixture, and Spherical KMeans.

Clause 41. The method of any one of clauses 33 to 40, wherein at leastsome of the plurality of features and the plurality of patient featuresare used in training the target prediction model to generate theplurality of trained target prediction models.

Clause 42. The method of any one of clauses 1 to 41, wherein informationon the first plurality of nucleic acids is obtained from a correspondingplurality of sequence reads derived from a respective patient sample bytargeted or whole transcriptome RNA sequencing.

Clause 43. The method of any one of clauses 1 to 42, wherein informationon the second plurality of nucleic acids is obtained from acorresponding plurality of sequence reads derived from a respectivepatient sample by targeted or whole transcriptome RNA sequencing.

Clause 44. The method of clause 42 or clause 43, wherein the wholetranscriptome sequencing comprises next-generation sequencing.

Clause 45. A method of any one of clauses 33 to 44, comprisingdisplaying, at least in part, the predictions on a graphical userinterface of a computing device.

Clause 46. The method of clause 45, wherein the predictions aredisplayed on the graphical user interface in association withinformation on at least some features from the plurality of featuresand/or the plurality of patient features.

Clause 47. The method of clause 46, comprising receiving, via thegraphical user interface, a request to display ranking informationassociated with the at least some features, the ranking informationcomprising a score associated with each feature of the at least somefeatures.

Clause 48. The method of clause 47, wherein the request comprises athreshold for scores associated with the features of the at least somefeatures, and wherein the method comprises displaying the information onthe at least some features based on the threshold.

Clause 49. The method of clause 48, wherein the information on the atleast some features comprises information on most influential genesand/or transcripts selected from the first plurality of nucleic acidsand/or the second plurality of nucleic acids.

EXAMPLES Example 1—Leave-One-Out Modeling Pipeline

Metastasis models were trained for each training subject in a trainingcohort of 162 subjects with available whole exome expression data.Briefly, the data from a single patient was removed from the trainingcohort and a model was fit. The process was then repeated for all thepatients within the cohort. The results of this is the establishment ofone model for every patient in the training cohort, and one predictionfor every patient in the training cohort. A Cox's proportional hazardmodel (CoxPHFitter) was used in a grid search searching over a set ofparameters for the model with the best concordance index scores.

Briefly, RNA expression data was normalized to transcripts per million(TPM) for all genes. The top 500 most correlated genes were thenidentified for each training run by Spearman Rank correlation. SingularValue Decomposition (SVD) was then performed for each training run,using the identified set of 500 genes. From the resulting SVD matrix,the first 40 components (columns) were then used as features in themodel.

From the 162 runs of the training pipeline (one for each subject in thetraining cohort), the total number of times each gene was identified inthe top 500 most correlated genes was determined. Table 2 lists thosegenes that were identified in the top 500 genes for at least 80 of the162 training runs.

TABLE 2 Most frequently identified genes, listed from most commonlyidentified to least commonly identified (column 1 = most frequentlyidentified; column 5 = least frequently identified). Column 1 Column 2Column 3 Column 4 Column 5 IGF1R TM2D1 RNF130 C10orf32 NDUFB8 ELOVL1C17orf49 ARHGAP6 YY1AP1 TSG101 ACP2 PGAM1 C5orf22 LAPTM4A TMCO1 NDUFA7SAMD13 CREB3 CHI3L1 CCND3 MT2A ENSG00000272617 MTFP1 PRKAA2 NELFCD CRIP2NQO2 IGFN1 NDUFV2 MANF ZNHIT1 NIPSNAP3A TMEM185A DNAJC15 SLC46A3 ATOX1CD63 FAM118A HCST GHRL GBA NINJ1 LILRA6 RTN3 CYBA PEF1 CD151 COX7BTIMM21 ABHD1 DHX37 HLA-DPA1 CBWD2 COPS8 HIST1H1C POLR2L TIMMDC1 SCAMP4USMG5 BCL2 PHPT1 CAP1 NFXL1 ISCA2 SKP1 ARF1 APOBR UMPS DHFR UBE2D1 MPC1MLANA C3orf35 DNAJA2 ZNF619 YARS NOMO3 YIPF3 NGRN PSPC1 SEC61B TMEM258IF130 RNF181 APPBP2 GPR108 MTRNR2L9 ATP8B1 ACTR1A RAB10 TPP1 PDE6GCLTCL1 UBL5 CMTM6 SERHL2 MRPL23 BTF3L4 OR2T5 PCGF6 TMX1 FUCA1 S100A1A3GALT2 OR51S1 CHRNG NDUFB1 CEPT1 ElF2B5 HSD11B1 CD4 UBE2L6 LMF2 OR4F4C9orf78 DPAGT1 TLN2 OR52D1 GABARAP FLRT1 ZMAT2 MAP4K3 FAM180B PEX16 CHADTAF10 TMEM107 C1orf85 HPD NPRL2 RPS19BP1 INO80 C2orf76 ANKRD61 SCARF1TUBA3E MRPL40 APOBEC3C SPCS3 HGD KRTCAP2 WBSCR22 MRO CTPS1 HIST2H2AA3PEBP1 B3GAT3 SCAMP3 RNF17 IFLTD1 ARIH1 CCDC36 PPARG PHC2 GEMIN2 RBFOX2TTC3 MFSD1 C1orf50 ZNF146 TMBIM6 SLC38A2 RPL41 SF3A3 GPR64 PIN1 SHISA4AKAP13 CENPA GNPTG TMED9 TRIM17 TMX2 PHKG1 WDR13 IFRD2 CATSPERG LYPLA2BECN1 POLD4 NDUFS2 S100A11 SERPINB6 TUBA3C MLF2 INIP CCL4L2 C7orf55MRPL27 ARRDC4 CDK5RAP3 NDUFC2 HYPK CCNH RIMBP3 SDHB DERL2 MT1X IL10RBRPS10 XPNPEP1 GNS SRGN TSEN15 NEU4 HSPB11 RNF121 PABPC1 LSM2 ZSWIM7C18orf32 GSTP1 ZBTB8OS DHRS7B LILRB1 MRPL38 DNAJC1 ENSG00000268643MAN2A2 ARF5 SRSF3 MTRNR2L10 AP1S1 CCDC53 SV2A SCPEP1 RNF220 ANKRD24 HEXAPLD3 DXO UROD VKORC1 PHYKPL LRRC34 RGS14 PIGK CYB5R1 EFCAB3 POPDC3 CD40VMA21 SYNGR4 SOX2 LMBRD1 TNFSF12 DRAP1 NDUFS3 SNAPC5 ENSG00000269871OR52B6 DDX41 PTRH1 CNPY2 HDDC3 IER3IP1 CCNDBP1 LPCAT3 FSCN2 PRPF39 FANCGSLC25A35 RFXANK OPCML FIS1 RPA2 SPRY2 UBR7 TMEM179B CPNE1 SOSTDC1ENSG00000206044 URI1 COQ10A NDFIP1 PSMG4 YPEL5 HPR EMD NKAIN1 CUTAMUM1L1 TSPAN9 SF3B6 CUEDC2 DHRS1 NDUFS7 CHRNB1 TRAK1 MRPL33 HSPA1A SATB1NUTM2G SSX2 CHURC1 HM13 RUVBL2 CEBPA AGAP9 PRDX1 C15orf38 LRTOMT CTSAPTTG1 MIF CA5B UIMC1 DDX54 RAB4B FPGT ENSG00000267954 C9orf89 TRAPPC1FPR1 PLEKHG4 UQCRC1 ALKBH4 BBS4 APOC2 HPSE SPCS1 DEPDC7 NEDD8 KPTN ASB1PEMT TRIT1 PIGY NMT1 GADD45A C19orf68 C9orf16 TUBB RAB5B DBNDD1 POLR3DH2AFZ CREBZF TSSK4 ZNF774 SLC10A1 ZMPSTE24 SC5D PDZD11 OCEL1 UQCR11CCDC137 DUSP18 RBFOX3 ANXA5 MLXIP GIMAP5 MECR COMMD1 GSK3A SCP2 LSM14AZNF587B FCER1G SCO1 TMEM208 PSAP AMT SNRPC ATXN2 SYCP3 HDAC10 CYSTM1ZNF337 POU5F1 C6orf48 AGA COX6A1 NHP2 STRA13 HEXB PRKACB NDUFS4 LAMTOR5ALKBH6 SRSF2 MDK CTSS A4GNT FASTKD3 GOLGA6L10 PHF12 ATP6V1E1 ALG3PPP2R2D TSC22D2 ENSG00000260537 CCL4L14 HIST2H2AA NUDT13 WDR45B NPNT HN1PGBD1 PRDX6 POGLUT1 ElF5A2 PFKM SUMO1 TMEM45B RABL2A MRPL22 DPM3 HFENKD2 RIN3 SYVN1 NDUFA4 MAP4K2 PGLYRP1 CAPZA1 TMEM41B CCL3 WLS SFN EIF3lMRPS36 PRICKLE4 COMMD9 LRP6 ATP2A1 TBC1D3G WIBG ARHGEF35 DAPK2 NPC2MAGOH NRM FBN3 FADS2 HMGN2 RNASEK MOB1A RTN4RL2 C8orf76 RBX1 CKLF POLR2JZNF462 TXNDC9 AMELX PFDN6 HATI CPA2 BYSL SPRED1 CNDP2 MRPS7 JKAMPENSG00000260342 DCAF7 HOXB7 ICAM2 DAD1 RAB17 CD82 TTC4 HIST1H2BC ZACNPSMB2 SLC35B1 GSTM4 ZNF259 TRIM2 EDRF1 GALNS C6orf226 SSBP1 ATG4APLEKHO1 MAN1B1 POP5 CCL4 KLRC4 TIMM23 MCCC1 SLC31A1 UBE2E1 HINT1 CXCL3CCL26 ERP44 SOWAHC ARHGAP28 ENTPD3 GNB2L1 MR1 HLA-DPB1 NKG7 VAMP5C5orf15 CTSD EMC6 ATF4 SNRNP25

Example 2—RNA Expression-Based Predication Model for Metastasis ofColorectal Cancer

RNA expression data obtained from tumor samples from 317 subjects withcolorectal cancer was used to train a Cox Proportional Hazard model fortime without metastasis of the cancer. Briefly, the inclusion criteriafor selection of the training subjects was: the subject had a primarycolorectal cancer diagnosis, RNA sequence data from tissue collectedfrom the primary colorectal tumor was available, the primary colorectalcancer was not a recurrence, subjects with positive margins wereexcluded, subjects with metastases recorded within 30 days of samplecollection were excluded, subjects staged with stage 4 cancer wereexcluded, patients without evidence of future metastasis must have atleast two years of follow-up medical records available, and fresh-frozentissue samples were excluded.

Briefly, a Cox Proportional Hazard model was trained by regression. RNAexpression features were prepared by log-transforming expression datafrom each tumor sample according to log(RNA+1). Zero variance columns ofthe RNA expression data were dropped. Features were cross-validated bydropping the lowest quartile of variance columns, keeping the top 100features most correlated (Spearman) with metastasis, and standardscaling. Stratified K-fold feature selection was performed using5-folds. Hyperparameters of the model were searched via grid search of aspline space defined as [0.2. 0.02, 0.002, 0.0002]. Model performancewas evaluated by repeated stratified k-fold cross-validation using10-folds repeated 10 times. A brief summary of model training is shownbelow:

Model Description:

-   -   CV framework:        -   Model Performance Evaluation:            -   RepeatedStratifiedKFold, 10 Folds, 10 Times.        -   Feature Selection:            -   StratifiedKFold, 5 Folds    -   Feature Generation        -   Overall            -   Drop Zero Variance Columns, log(RNA+1) Transform        -   Cross-Validated:            -   Drop Lowest Quartile of Variance Columns            -   Keep Top 100 Features Most correlated (Spearman) with                metastasis,            -   Standard Scaling    -   Model type: Cox Proportional Hazard Regression    -   Hyperparameter search method and space        -   Method: Grid Search        -   Space: Spline=[0.2, 0.02, 0.002, 0.0002]

Survival curves of the model showing predicted metastasis-free survivalfor high-risk patients (1902) and low-risk patients (1904) areillustrated in FIG. 19A. The hazard ratio comparing the predictedhigh-risk group to the low-risk group is 5.3 (3.73-8.17).

The risk status for a patient was determined using a risk scorethreshold, which can be selected between 0% and 100% of the totalpossible model output (e.g., 0 to 1 for a model in which 1 represents100% chance of metastasis). In this example, the risk threshold wasdetermined by using the percentile from ranked predictions thatcorresponds to the metastasis rate of the training cohort. That is, if74% of patients in the cohort experienced metastasis, the risk thresholdis set as the predicted probability at the 26^(th) percentile. Forinstance, FIG. 19B illustrates a histogram of model scores output froman entire training cohort in which 74% of the training subjectsexperienced metastasis. The model score at the 26^(th) percentile(between 0.3 and 0.4) is set as the risk threshold.

Example 3—Predication Model for Metastasis of Colon and RectosigmoidJunction Cancers

RNA expression data obtained from tumor samples from 173 subjects withcolon or rectosigmoid junction cancer was used to train a CoxProportional Hazard model for time without metastasis of the cancer.Briefly, the inclusion criteria for selection of the training subjectswas: the subject had a primary colon or rectosigmoid junction cancerdiagnosis, RNA sequence data from tissue collected from the primarycolorectal tumor was available, the primary colorectal cancer was not arecurrence, subjects with positive margins were excluded, subjects withmetastases recorded within 30 days of sample collection were excluded,subjects staged with stage 4 cancer were excluded, patients withoutevidence of future metastasis must have at least two years of follow-upmedical records available from members of the subject's cancer treatmentteam, fresh-frozen tissue samples were excluded, subjects staged withstage 1 cancer were excluded, non-adenocarcinomas were excluded, samplestaken from biopsies were excluded (only samples obtained from resectionswere used), subjects with metastases recorded within 90 days of samplecollection or who were determined to be likely metastatic at the time ofsample collection were excluded, and subjects undergoing systematictreatment for an additional primary cancer within two years of the coloncancer diagnosis were excluded.

Metastasis models were trained for each training subject in a trainingcohort of 162 subjects with available whole exome expression data.Briefly, the data from a single patient was removed from the trainingcohort and a model was fit. The process was then repeated for all thepatients within the cohort. The results of this is the establishment ofone model for every patient in the training cohort, and one predictionfor every patient in the training cohort. A Cox's proportional hazardmodel (CoxPHFitter) was used in a grid search searching over a set ofparameters for the model with the best concordance index scores.

Briefly, RNA expression data was normalized to transcripts per million(TPM) for all genes. The top 500 most correlated genes were thenidentified for each training run by Spearman Rank correlation. SingularValue Decomposition (SVD) was then performed for each training run,using the identified set of 500 genes. From the resulting SVD matrix,the first 40 components (columns) were then used as features in themodel. A brief summary of model training is shown below:

Model Description:

-   -   CV framework:        -   Model Performance Evaluation:            -   LeaveOneOut: 173 Splits        -   Feature Selection:            -   SurvivalSplitter, 5 Splits    -   Feature Generation:        -   Overall:            -   Drop Column with Zero Variance        -   Cross-Validated            -   Drop Lowest Quartile of Variance Columns            -   Keep Top 500 Features Most correlated (Spearman) with                metastasis                -   StandScaler            -   Singular Value Decomposition, Keeping First 40 Columns            -   StandardScaler    -   Model type:        -   Cox Proportional Hazard Regression    -   Hyperparameter search method and space:        -   None

Survival curves of the model showing predicted metastasis-free survivalfor high-risk patients (2002) and low-risk patients (2004) areillustrated in FIG. 20. The hazard ratio comparing the predictedhigh-risk group to the low-risk group is 2.58 (1.74-3.82).

The risk status for a patient was determined using a risk scorethreshold, which can be selected between 0% and 100% of the totalpossible model output (e.g., 0 to 1 for a model in which 1 represents100% chance of metastasis). In this example, the risk threshold wasdetermined by using the percentile from ranked predictions thatcorresponds to the metastasis rate of the training cohort. That is, if74% of patients in the cohort experienced metastasis, the risk thresholdis set as the predicted probability at the 26^(th) percentile. Forinstance, FIG. 19B illustrates a histogram of model scores output froman entire training cohort in which 74% of the training subjectsexperienced metastasis. The model score at the 26^(th) percentile(between 0.3 and 0.4) is set as the risk threshold.

From the 173 runs of the training pipeline (one for each subject in thetraining cohort), the total number of times each gene was identified inthe top 500 most correlated genes was determined. Table 3 lists all thegenes identified in the top 500 most correlated genes and denotes howmany times the gene was identified in that list.

TABLE 3 Frequency of genes appearing in the top 500 most correlatedgenes. # of Gene times Name used MBNL2 173 NAPB 173 C1orf111 173 MPZL3173 MR1 173 MROH5 173 MRPL22 173 MT1X 173 MT2A 173 MTHFS 173 C1QBP 173NAA16 173 NAPG 173 MPHOSPH8 173 C19orf80 173 NCF1 173 C15orf65 173NDFIP2 173 NEURL1B 173 NKD2 173 NKG7 173 NLRP11 173 NR1I2 173 C11orf84173 C1orf85 173 MPDU1 173 NRP1 173 LRCH1 173 CARS2 173 CARKD 173LGALS3BP 173 LHPP 173 LILRA6 173 LIPA 173 LITAF 173 LMAN1 173 C9orf50173 C9orf116 173 C8orf33 173 MICAL3 173 LRRTM4 173 C6orf25 173 LY6G6D173 LY6G6F 173 MANBA 173 MAPK12 173 MAZ 173 C2orf54 173 C20orf196 173MEST 173 NR6A1 173 NUTM2B 173 KSR2 173 ASAH1 173 PRSS56 173 PSAP 173PTAFR 173 PTTG2 173 QSOX1 173 R3HDML 173 RAB3A 173 RALY 173 ASIP 173RAP2A 173 RBM26 173 PROM1 173 RDH10 173 RDH11 173 REPS2 173 ARID3A 173RNF144B 173 ARHGEF35 173 RPA2 173 RPGR 173 RSL1D1 173 RTDR1 173 PROSER1173 PRLR 173 CPNE6 173 PFKP 173 OTX1 173 P2RX4 173 PABPC3 173 PAPD5 173PBX2 173 PCCB 173 PCID2 173 PDIA3 173 PDS5B 173 PEMT 173 BCORL1 173PRDX3 173 BCL11A 173 PIGL 173 PKP1 173 ATP7B 173 ATP11A 173 ASXL1 173POLR1D 173 PPP4R1 173 PRB1 173 PRB4 173 CASS4 173 CCL26 173 APCDD1L 173ELF3 173 DNAH6 173 COBLL1 173 DNAJC28 173 DNASE2 173 DPEP1 173 DRD2 173DUSP16 173 ECI2 173 CNDP2 173 EIF5A 173 EMP3 173 DLX3 173 ENGASE 173CLEC4E 173 F3 173 F7 173 CLDN7 173 FAM57A 173 FARP1 173 FBXL3 173 FBXO16173 CITED1 173 DNAAF3 173 DLEU7 173 FBXO6 173 CUL4A 173 CPVL 173 CPN1173 CPD 173 CRYBB3 173 CSF3R 173 CTSB 173 CTSD 173 CTSE 173 CTSL 173CTSS 173 CXCL16 173 DIAPH3 173 CXCR2 173 COQ2 173 CYBA 173 CYP2F1 173COPS3 173 CYP4F2 173 CYSTM1 173 DACH1 173 DGKH 173 DHRS7 173 CHRFAM7A173 FCER1G 173 KRT40 173 CD68 173 HBEGF 173 HINT2 173 CD84 173 HOXA2 173HOXA6 173 HPSE 173 HSPA1A 173 HSPH1 173 HUNK 173 IF130 173 IHH 173 GZF1173 IL1R2 173 IL22RA1 173 IPO5 173 IQCG 173 IZUMO2 173 KBTBD6 173KIAA0226L 173 KIF16BL 173 CD14 173 KRT39 173 H2AFZ 173 GRN 173 FCGR1A173 GAS6 173 FCGR2A 173 FCGR3B 173 FCN3 173 FECH 173 FOXH1 173 FPR1 173FRRS1 173 FTCD 173 FUCA1 173 G6PC3 173 GCM2 173 CDHR1 173 CEP192 173GDPD5 173 CENPJ 173 CELA2B 173 GJC3 173 GLIPR2 173 CDX2 173 GNS 173GOLGA6L2 173 GPR143 173 SCG5 173 BSG 173 STARD13 173 ACADSB 173 YAP1 173WNT11 173 STK24 173 STK35 173 STMND1 173 STON2 173 WDR76 173 SYNJ2 173TCN1 173 SYNPR 173 VWA8 173 TAF4 173 AIRE 173 TASP1 173 TBC1D4 173 TCF20173 SSUH2 173 AKR1B1 173 ALDH3A2 173 SPRY2 173 SLC9A7 173 ZNF133 173SLX4IP 173 ZMYND8 173 ANKRD10 173 SNTB1 173 SNX32 173 ALX3 173 ZIC2 173SP140L 173 ZFHX3 173 SPATA13 173 ZC3H13 173 ZBTB10 173 SPIN2B 173 AGTRAP173 TEAD4 173 ZNF263 173 TNFSF13 173 ADNP 173 TSR1 173 TMPRSS11A 173ADCY10 173 TMPRSS11D 173 TMPRSS6 173 TMTC4 173 TNNC2 173 ACP5 173 ADM173 TSPO2 173 TRIM13 173 TSPAN7 173 TSC22D1 173 ADCY3 173 TSPAN13 173TTYH1 173 TUBG1 173 TMEM61 173 TMEM254 173 VKORC1 173 VAPA 173 TFDP1 173USP14 173 ACSL6 173 ACSS1 173 UPF3A 173 UMODL1 173 ANXA10 173 ULK2 173TMEM139 173 UFM1 173 TMEM178A 173 AGA 173 TXNDC17 173 ZNF232 173 TSPAN12173 SESN1 173 ABHD12 173 SLC31A1 173 SLC17A7 173 SLC16A14 173 SLC25A35173 SHROOM4 173 AADAT 173 ABHD3 173 SHMT1 173 ZNF396 173 SERPINB6 173ZNRF2 173 ANO9 173 ZNF3 173 SIGLEC11 173 SIGLEC8 173 SIM2 173 FAM151A172 TSPY1 172 OSBP2 172 C18orf42 172 SLC25A11 172 APOL1 172 YWHAE 172KIAA1279 172 SYDE2 172 ZNRF3 172 DUSP8 172 KCTD17 172 SERPINB1 172 KCNRG172 ARHGEF7 172 IZUMO3 172 GRTP1 172 MGA 172 MINK1 172 DAPK2 172 SLC6A12172 SNX10 172 SLC35B1 172 FGF3 172 AGR2 172 RIN2 172 CHAD 172 SLC50A1172 DHRS7B 172 C5orf15 172 HPS4 171 HGD 171 LGR5 171 RNF17 171 SLC39A5171 PRB3 171 PAFAH1B1 171 CPT2 171 COX10 171 BEX2 171 NUP88 171 PCK1 171SCGB2A2 171 LAMTOR5 171 MPC1 171 CDC16 171 CCL4L1 171 IGF1R 171 C17orf49171 S100A6 171 ARGLU1 171 OOEP 171 RBM39 171 PLA2G6 170 DHRS12 170DNAJC1 170 FRMD3 170 VPS53 170 SERPINA1 170 PER2 170 KLRF2 169 LAG3 169PPP1R15A 169 FER1L5 169 PLD2 169 MUC15 168 WRAP53 168 VSIG10 168 IGSF2168 ATP6V1B2 167 CRHR2 167 RAP1GAP2 167 VNN2 166 DDX60 166 C12orf75 166ZNF551 166 FRMD1 166 LAPTM5 166 BTBD18 166 ZSWIM3 165 TPSAB1 165 CTSC165 UBE2L6 165 GPANK1 165 ZNF48 164 HSD3B7 164 MICU1 164 SUGP2 164 DOCK9164 MRPS7 164 USP54 164 RMND5A 164 P4HB 164 SLC22A3 163 PABPC1 162TMEM211 162 TUBGCP3 162 CSNK2A2 162 KDELR3 162 RAC3 162 SH3D19 160 RILP160 BLVRB 160 LRRC71 159 AK1 159 VAMP5 159 CD59 158 NCOA6 158 HLA-G 158AQP12B 158 ZMYM2 157 NCF2 157 SOX14 157 PAGE2B 157 ABHD6 157 IFNA1 156OSCP1 156 FAM211B 156 SFXN1 155 GPR18 155 HEATR6 154 SPIRE2 154 DCUN1D2154 DYX1C1 153 ZNF852 152 KLRD1 152 CPSF4L 152 TEX9 152 NXPH1 152 MPP3151 CD82 151 HDHD2 149 CLDN16 148 CLCF1 148 CLHC1 148 RPS27L 148 N4BP2L2148 FAM83C 148 TMCO1 147 SLC39A4 146 RNF113B 146 FPR2 145 ADPRM 144MAP2K3 144 GLTPD2 143 ASPDH 143 PLEKHG6 142 ALDH6A1 140 MYCBP2 140 CAMKV139 TMCO3 139 FBL 137 EMR3 137 ZBTB44 137 TIMP1 136 HOXA5 135 ARL15 135IGFN1 134 SLC30A2 133 SETD6 133 ADIRF 133 MOCOS 132 HFE 131 ZNF506 131MRPL38 130 FAM107B 128 COL2A1 128 RPL3 124 SMIM14 124 ANTXR2 122 DBX1121 CRYZ 120 RNASEH2B 119 SOX4 119 TOR1B 119 CTH 118 YIPF1 118 NT5M 116URB1 113 TPP1 113 ARHGEF19 113 PSMG2 110 ZNF730 109 TCF7 107 ZFP28 107RNF217 105 HNF4A 104 CLCN2 103 AGR3 101 NEK3 97 ATG9B 96 ALDH1L1 82 GMDS81 ABHD10 74 CTTNBP2 55 PSMA5 53 HCST 47 IDH1 47 PRRT2 46 UBR5 45RBBP8NL 45 FCGR3A 43 ZFC3H1 41 MDH1 41 ANKRD45 40 SPATA2 39 NKD1 38CDCP1 38 CD74 37 RPS12 36 PLA2G12B 36 TRAF5 35 C3orf33 35 ADRBK2 35AKAP1 35 DOCK8 33 TSC22D4 33 SECTM1 32 TLE2 30 RRS1 30 OR56B4 30 LMTK229 NFATC2 29 TERT 29 UGGT2 29 CYP2B6 29 NUTM2G 29 PNP 28 ABLIM2 27 CSF127 PRSS33 27 RBM12B 27 REG4 26 LMTK3 26 NDUFC2 26 SLC6A8 25 TNNT1 25 MRO25 PPARA 24 GRHL3 24 THRA 24 CDK18 23 RPL14 23 MYL12A 23 ACO1 23 APOO 22C1orf195 22 TAS2R43 22 TRAF3IP1 22 SPDYC 22 BTD 21 ZNF587 20 POU5F1 20STAMBPL1 20 S100A16 20 SMAD7 19 TFAP4 19 NOXO1 19 RGL4 19 ADAM21 19 SCIN19 SDR42E1 19 TMEM102 19 CDHR3 18 AZGP1 18 SMOX 18 OSER1 18 CAMK2D 17MORN1 17 SRR 17 SHROOM2 17 C7orf55-LUC7L2 17 AMELX 16 TM4SF19 16 NES 16FIGF 16 MOCS2 16 RALGAPA2 15 TK2 15 MTRF1 15 GTF2IRD1 15 PALM3 15 ENG 15C22orf23 15 ZNF646 15 TJP2 15 PLCB4 14 TMEM62 14 TGIF2LX 14 CEP76 13OVOL2 13 SERPIND1 13 LGR6 13 GUCY2C 13 UBOX5 13 LARP6 12 SCPEP1 12 ETFDH12 EMR2 12 MAMSTR 12 GAD1 12 GPR183 12 ZNF251 12 MYOM3 12 HVCN1 11TMEM150B 11 LGMN 11 DENND2D 11 IF135 11 KANK1 11 FXYD7 11 FBF1 11FAM186A 11 SYCE2 11 EPHA8 11 GAPDHS 11 IVD 10 HAL 10 URGCP 10 CHD7 10ZKSCAN1 10 BPHL 9 RPS7 9 KPTN 9 BRI3 9 SYT7 9 ZNF287 9 SAT2 9 CHPT1 9CSNK2A3 8 SLC39A6 8 RARG 8 GSTM4 8 KNOP1 8 EID1 8 MDK 8 TMEM110 8 LRP6 8MAP2K4 8 TSACC 8 OASL 7 HS6ST2 7 PCDHGA11 7 TYROBP 7 RCOR2 7 KRT4 7MIS12 7 PGD 7 EPOR 7 CEL 6 RRAD 6 FAT3 6 MACROD1 6 PNPLA3 6 MLPH 6MOGAT3 6 ARSA 6 CMTM4 6 OR2A7 6 COX7A2 5 C1QC 5 CD244 5 ANKRD27 5 ASS1 5A3GALT2 5 RPL7 5 RPS3 5 TRIM29 5 SQRDL 5 SRCRB4D 5 SORL1 5 LIPM 5 RBP5 5SOAT1 5 PPP1R32 5 FGFR2 5 OSBPL2 5 FAM101A 5 HMGCL 5 FAM162A 5 PLA2G4A 4CD27 4 HLTF 4 NR2E1 4 TGM1 4 MPPE1 4 MESP2 4 CFTR 4 SSX4 4 PLK5 4RPS10-NUDT3 4 FCAMR 4 VMO1 4 RPTN 3 GJA5 3 LCP2 3 CNOT1 3 RBBP8 3KRTAP10-8 3 RHOC 3 CCR9 3 OCEL1 3 KCNIP2 3 GALK1 3 ERBB3 3 CYB5D2 3VPS4B 3 PLCH1 3 MPL 3 TMEM63C 3 BCL3 3 PMP22 3 ACVR1B 3 TUFT1 3 GJC2 3NCK2 3 MTIF3 2 RIIAD1 2 RNASET2 2 RANBP6 2 NCKAP5 2 NCOA2 2 ASCL5 2SERPINB7 2 PCDH20 2 PHKG1 2 ANXA5 2 PLA2G2A 2 ATP5A1 2 TFAP2A 2 SS18 2TMEM220 2 ZNF614 2 DIRC2 2 ZNF326 2 ZDHHC20 2 WNT10A 2 FBXO41 2 URI1 2GEMIN4 2 CDR2 2 GOLIM4 2 TOP1MT 2 GPR182 2 HAVCR2 2 MAP7 2 TMEM237 2LPCAT1 2 INTS6 2 C7orf55 2 KLKB1 2 L3HYPDH 2 CAMTA2 2 TCIRG1 2 LPIN2 2SKIL 1 ZSCAN21 1 SYN3 1 ZNF507 1 TAS2R20 1 SPCS1 1 WARS 1 TSEN2 1 UPP1 1TSPY4 1 SH3BP4 1 RNF6 1 SEC61A2 1 SMCHD1 1 SLC5A6 1 TMEM205 1 SEH1L 1AFTPH 1 CRB2 1 BTN2A2 1 RNF215 1 CISD1 1 HNRNPA1L2 1 CDADC1 1 HIRA 1GPR157 1 GLS 1 GDPD1 1 FER1L6 1 FBXO44 1 CHRNG 1 CISD2 1 KLK8 1 FAM25G 1FAM105A 1 ENTPD6 1 EIF4A1 1 EID3 1 DNAJC21 1 CYP4F12 1 CXXC5 1 CSDC2 1IGSF3 1 LACTB 1 ARRB2 1 CRLS1 1 RAB27B 1 ATP4B 1 PILRA 1 PHOSPHO2 1 PEX51 PARP2 1 BRCA2 1 OVGP1 1 NUDT6 1 NQO2 1 LGALS9 1 NARS 1 MRAP 1 MPI 1MERTK 1 ZSWIM4 1 MAPRE3 1 LUC7L 1 LRRN3 1 LRRC16B 1 ZSWIM7 1

Example 4—Predication Model for Metastasis of Colon and RectosigmoidJunction Cancers

The model presented in Example 3 was prepared using expression dataobtained using multiple RNA expression pipelines. To evaluate theresults of model training on data obtained using a single RNA expressionpipeline, a subset of 106 of the 173 training subjects used in Example3, for which RNA expression data was generated using the same RNAexpression pipeline, was used to train a second model following the sameprocedure as in Example 3.

Survival curves of the model showing predicted metastasis-free survivalfor high-risk patients (2102) and low-risk patients (2104) areillustrated in FIG. 21. The hazard ratio comparing the predictedhigh-risk group to the low-risk group is 5.20 (2.9-9.32).

The risk status for a patient was determined using a risk scorethreshold, which can be selected between 0% and 100% of the totalpossible model output (e.g., 0 to 1 for a model in which 1 represents100% chance of metastasis). In this example, the risk threshold wasdetermined by using the percentile from ranked predictions thatcorresponds to the metastasis rate of the training cohort. That is, if74% of patients in the cohort experienced metastasis, the risk thresholdis set as the predicted probability at the 26^(th) percentile. Forinstance, FIG. 19B illustrates a histogram of model scores output froman entire training cohort in which 74% of the training subjectsexperienced metastasis. The model score at the 26^(th) percentile(between 0.3 and 0.4) is set as the risk threshold.

Example 5—RNA Expression-Based Predication Model for Metastasis ofColorectal Cancer

RNA expression data obtained from tumor samples from 173 subjects withcolon or rectosigmoid junction cancer was used to train a CoxProportional Hazard model for time without metastasis of the cancer.Briefly, the inclusion criteria for selection of the training subjectswas: the subject had a primary colon or rectosigmoid junction cancerdiagnosis, RNA sequence data from tissue collected from the primarycolorectal tumor was available, the primary colorectal cancer was not arecurrence, subjects with positive margins were excluded, subjects withmetastases recorded within 30 days of sample collection were excluded,subjects staged with stage 4 cancer were excluded, patients withoutevidence of future metastasis must have at least two years of follow-upmedical records available from members of the subject's cancer treatmentteam, fresh-frozen tissue samples were excluded, subjects staged withstage 1 cancer were excluded, non-adenocarcinomas were excluded, samplestaken from biopsies were excluded (only samples obtained from resectionswere used), subjects with metastases recorded within 90 days of samplecollection or who were determined to be likely metastatic at the time ofsample collection were excluded, and subjects undergoing systematictreatment for an additional primary cancer within two years of the coloncancer diagnosis were excluded.

Briefly, RNA expression data was normalized to transcripts per million(TPM) for all genes. The top 100 most correlated genes were thenidentified for each training run by Spearman Rank correlation. The datafrom a single patient was removed from the training cohort and a modelwas fit. The process was then repeated for all the patients within thecohort. The results of this is the establishment of one model for everypatient in the training cohort, and one prediction for every patient inthe training cohort. A brief summary of model training is shown below:

Model description:

-   -   CV framework:        -   Model Performance Evaluation:            -   LeaveOneOut: 173 Splits        -   Feature Selection:            -   SurvivalSplitter, 5 Splits    -   Feature Generation:        -   Overall:            -   Drop Column with Zero Variance        -   Cross-Validated            -   Drop Columns with a Low Expression (Median Expression                <0)            -   Keep Top 100 Features Most correlated (Spearman) with                metastasis            -   PowerTransforms            -   Kernel PCA (Cosine Kernel) Keep First 20 Columns            -   StandardScaler    -   Model type:        -   Cox Proportional Hazard Regression    -   Hyperparameter search method and space:        -   None

Survival curves of the model showing predicted metastasis-free survivalfor high-risk patients (2202) and low-risk patients (2204) areillustrated in FIG. 22. The hazard ratio comparing the predictedhigh-risk group to the low-risk group is 4.21 (2.81-6.32).

The risk status for a patient was determined using a risk scorethreshold, which can be selected between 0% and 100% of the totalpossible model output (e.g., 0 to 1 for a model in which 1 represents100% chance of metastasis). In this example, the risk threshold wasdetermined by using the percentile from ranked predictions thatcorresponds to the metastasis rate of the training cohort. That is, if74% of patients in the cohort experienced metastasis, the risk thresholdis set as the predicted probability at the 26^(th) percentile. Forinstance, FIG. 19B illustrates a histogram of model scores output froman entire training cohort in which 74% of the training subjectsexperienced metastasis. The model score at the 26^(th) percentile(between 0.3 and 0.4) is set as the risk threshold.

Example 6—Multi-Modal Modeling for Metastasis of Colorectal Cancer

To evaluate the effect of combining feature types on the ability tomodel metastases, a stepladder analysis, where features were added tosubsequent model builds, was performed. Briefly, pathology data, tumorimaging data, DNA sequencing data, and RNA expression data was obtainedfrom tumor samples from 146 subjects with colon or rectosigmoid junctioncancer, and used to train a series of Cox Proportional Hazard model fortime without metastasis of the cancer. Briefly, the inclusion criteriafor selection of the training subjects was: the subject had a primarycolon or rectosigmoid junction cancer diagnosis, RNA sequence data fromtissue collected from the primary colorectal tumor was available, theprimary colorectal cancer was not a recurrence, subjects with positivemargins were excluded, subjects with metastases recorded within 30 daysof sample collection were excluded, subjects staged with stage 4 cancerwere excluded, patients without evidence of future metastasis must haveat least two years of follow-up medical records available from membersof the subject's cancer treatment team, fresh-frozen tissue samples wereexcluded, subjects staged with stage 1 cancer were excluded,non-adenocarcinomas were excluded, samples taken from biopsies wereexcluded (only samples obtained from resections were used), subjectswith metastases recorded within 90 days of sample collection or who weredetermined to be likely metastatic at the time of sample collection wereexcluded, and subjects undergoing systematic treatment for an additionalprimary cancer within two years of the colon cancer diagnosis wereexcluded.

A brief summary of data preprocessing and model training is shown below:

Model Description:

-   -   CV framework:        -   Model Performance Evaluation:            -   LeaveOneOut: 146 Splits        -   Feature Selection:            -   StratifiedKFold: 5 Splits, stratified on 2 year                metastasis binary label    -   Preprocessing:        -   Binary encode gender        -   Ordinal encode stage, N stage, and T stage as corresponding            integer in [0, 1, 2, 3, 4]        -   Binary encode histopath grade as 0=low grade, 1=high grade        -   RNA gene expression transformed as log 2(TPM)        -   Binary encode DNA features as 1=somatic pathogenic gene            variant, 0=no somatic pathogenic gene variant        -   Drop any columns missing all values or with zero variance    -   Feature selection and inline preprocessing:        -   RNA features            -   Drop genes below 25th percentile of variance            -   Select top 500 genes by Spearman correlation with target                observed (uncensored) duration until metastasis            -   Apply standard scaling (z-score) transformation        -   DNA features            -   Drop gene variants below 25th percentile of variance            -   Extract top 8 most important features as ranked by                fitting a RandomForestClassifier (2000 estimators, max                depth 2, max samples 0.8) on the 2 year binary label for                metastasis        -   Imaging features            -   Select top 100 imaging features by Spearman correlation                with target observed (uncensored) duration until                metastasis            -   QuantileTransform features into a uniform distribution                in range [−1, 1]        -   Clinical features            -   Apply standard scaling (z-score) transformation to age                at sample collection; mean fill any missing age            -   Fill any missing T stage and N stage using                k-NearestNeighbors based on all other stage data            -   Mean encode histology and histopath grade, such that                each category value is replaced by the mean duration                until metastasis within each value group            -   Apply standard scaling (z-score) transformation to all                clinical features    -   Dimensionality reduction:        -   RNA            -   Singular Value Decomposition of standard scaled top 500                duration correlated genes from feature selection; retain                top 20 (largest magnitude singular values) components;                mean center each SVD component        -   DNA            -   No dimensionality reduction        -   Imaging            -   Singular Value Decomposition of uniform distribution                features in range [−1, 1]; retain top 20 (largest                magnitude singular values) components; mean center each                SVD component        -   Clinical            -   No dimensionality reduction    -   Model type:        -   Cox Proportional Hazards model (lifelines python package)            -   Baseline hazard predicted by spline            -   L2 coefficient regularization            -   Apply power calibration transformation of predictions                such that the baseline survival probability equals 0.5                at 2 years    -   Hyperparameter search method and space:        -   Grid search maximizing the c-index of the concatenation of            out-of-fold predictions in the StratifiedKFold inner cross            validation loop:            -   L2 regularization parameter: [0.2, 0.02, 0.002, 0.0002]

Four predictive models were trained: (i) clinical data only, (ii)clinical data and imaging data, (iii) clinical data, imaging data, andDNA somatic mutation data, and (iv) clinical data, imaging data, DNAsomatic mutation data, and RNA expression data. As shown in Table 4,below, evaluation of the four models indicates that there is an additivebenefit to the addition of each type of data to the predictive model,across all three metrics considered.

TABLE 4 Analysis of Cox Proportional Hazards predictive of metastases incolon or rectosigmoid junction cancer. Clinical + Clinical + Clinical +Imaging + Clinical Imaging Imaging + DNA DNA + RNA Hazard 1.41 1.71 1.912.76 ratio (0.98-2.02) (1.18-2.49) (1.32-2.78) (1.88-4.05) C-index 0.5420.5855 0.596 0.641 AUROC 0.568 0.678 0.718 0.756

Example 7—RNA Expression-Based Predication Model for Metastasis ofNon-Small Cell Lung Cancer (NSCLC)

RNA expression data obtained from tumor samples from subjects with NSCLCcan be used to train a Cox Proportional Hazard model for time withoutmetastasis of the cancer. In a first training, inclusion criteriaincludes: the subject has a primary lung cancer diagnosis with NSCLChistology, RNA sequence data from tissue collected from the primary lungtumor was available, subjects with metastases recorded within 90 days ofsample collection were excluded, subjects staged with stage 4 cancerwere excluded, patients without evidence of future metastasis must haveat least two years of follow-up medical records available, andfresh-frozen tissue samples were excluded.

In a second training, inclusion criteria includes: the subject has aprimary lung cancer diagnosis with NSCLC histology, RNA sequence datafrom tissue collected from the primary lung tumor was available,subjects with metastases recorded within 90 days of sample collectionwere excluded, subjects staged with stage 3 or stage 4 cancer wereexcluded, patients without evidence of future metastasis must have atleast two years of follow-up medical records available, and fresh-frozentissue samples were excluded, patients undergoing systemic treatment foran additional primary cancer within 2 years of NSCLC diagnosis (exceptfor hormone therapy) were excluded, samples taken from recurrences wereexcluded (only include samples taken from initial diagnosis), patientswith squamous histologies were excluded, patients who did not receiveresections were excluded, patients with positive surgical margins wereexcluded, and EGFR positive patients were excluded.

Some trainings are performed according to the methodologies outlined inany one of Examples 1-5. Other trainings are performed using other knownmethodologies for training survival models.

Example 8—Multi-Modal Modeling for Metastasis of Non-Small Cell LungCancer (NSCLC)

Various combinations of pathology data, tumor imaging data, DNAsequencing data, and RNA expression data are used to train a model,e.g., a survival model, for predicting metastasis. inclusion criteriaincludes: the subject has a primary lung cancer diagnosis with NSCLChistology, RNA sequence data from tissue collected from the primary lungtumor was available, subjects with metastases recorded within 90 days ofsample collection were excluded, subjects staged with stage 4 cancerwere excluded, patients without evidence of future metastasis must haveat least two years of follow-up medical records available, andfresh-frozen tissue samples were excluded.

In a second training, inclusion criteria includes: the subject has aprimary lung cancer diagnosis with NSCLC histology, RNA sequence datafrom tissue collected from the primary lung tumor was available,subjects with metastases recorded within 90 days of sample collectionwere excluded, subjects staged with stage 3 or stage 4 cancer wereexcluded, patients without evidence of future metastasis must have atleast two years of follow-up medical records available, and fresh-frozentissue samples were excluded, patients undergoing systemic treatment foran additional primary cancer within 2 years of NSCLC diagnosis (exceptfor hormone therapy) were excluded, samples taken from recurrences wereexcluded (only include samples taken from initial diagnosis), patientswith squamous histologies were excluded, patients who did not receiveresections were excluded, patients with positive surgical margins wereexcluded, and EGFR positive patients were excluded.

Some trainings are performed according to the methodologies outlined inExample 6. Other trainings are performed using other known methodologiesfor training survival models.

CONCLUSION

Some portions of the preceding detailed descriptions have been presentedin terms of algorithms and symbolic representations of operations ondata bits within a computer memory. These algorithmic descriptions andrepresentations are the ways used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of operations leading to adesired result. The operations are those requiring physicalmanipulations of physical quantities. Usually, though not necessarily,these quantities take the form of electrical or magnetic signals capableof being stored, combined, compared, and otherwise manipulated. It hasproven convenient at times, principally for reasons of common usage, torefer to these signals as bits, values, elements, symbols, characters,terms, numbers, or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the above discussion, itis appreciated that throughout the description, discussions utilizingterms such as “identifying” or “providing” or “calculating” or“determining” or the like, refer to the action and processes of acomputer system, or similar electronic computing device, thatmanipulates and transforms data represented as physical (electronic)quantities within the computer system's registers and memories intoother data similarly represented as physical quantities within thecomputer system memories or registers or other such information storagedevices.

The present disclosure also relates to an apparatus for performing theoperations herein. This apparatus may be specially constructed for theintended purposes, or it may include a general purpose computerselectively activated or reconfigured by a computer program stored inthe computer. Such a computer program may be stored in a computerreadable storage medium, such as, but not limited to, any type of diskincluding floppy disks, optical disks, CD-ROMs, and magnetic-opticaldisks, read-only memories (ROMs), random access memories (RAMs), EPROMs,EEPROMs, magnetic or optical cards, or any type of media suitable forstoring electronic instructions, each coupled to a computer system bus.

The algorithms and displays presented herein are not inherently relatedto any particular computer or other apparatus. Various general purposesystems may be used with programs in accordance with the teachingsherein, or it may prove convenient to construct a more specializedapparatus to perform the method. The structure for a variety of thesesystems will appear as set forth in the description below. In addition,the present disclosure is not described with reference to any particularprogramming language. It will be appreciated that a variety ofprogramming languages may be used to implement the teachings of thedisclosure as described herein.

The present disclosure may be provided as a computer program product, orsoftware, that may include a machine-readable medium having storedthereon instructions, which may be used to program a computer system (orother electronic devices) to perform a process according to the presentdisclosure. A machine-readable medium includes any mechanism for storinginformation in a form readable by a machine (such as a computer). Forexample, a machine-readable (such as computer-readable) medium includesa machine (such as a computer) readable storage medium such as a readonly memory (“ROM”), random access memory (“RAM”), magnetic disk storagemedia, optical storage media, flash memory devices, etc.

In the foregoing specification, implementations of the disclosure havebeen described with reference to specific example implementationsthereof. It will be evident that various modifications may be madethereto without departing from the broader spirit and scope ofimplementations of the disclosure as set forth in the following claims.The specification and drawings are, accordingly, to be regarded in anillustrative sense rather than a restrictive sense.

1. A method for predicting metastasis of a cancer in a subject,comprising: at a computer system having one or more processors, andmemory storing one or more programs for execution by the one or moreprocessors: (A) obtaining, in electronic format, a plurality of at least10,000 sequence reads, wherein the plurality of sequence reads isobtained for a plurality of RNA molecules from a sample of the cancerobtained from the subject (B) determining from the plurality of at least10,000 sequence reads, a plurality of data elements for the subject'scancer comprising: a first set of sequence features comprising abundancevalues for the expression of a plurality of at least 30 genes in thesample of the cancer obtained from the subject; and (C) applying, to theplurality of data elements for the subject's cancer comprising the firstset of sequence features comprising abundance values for the expressionof the plurality of at least 30 genes, one or more models that arecollectively trained to provide a respective one or more indications ofwhether the cancer will metastasize in the subject, thereby predictingwhether the cancer will metastasize.
 2. The method of claim 1, whereinthe plurality of data elements further comprises one or more personalcharacteristics about the subject selected from the group consisting ofage, gender, and race and wherein the (C) applying includes applying theone or more personal characteristics about the subject to the one ormore models.
 3. The method of claim 1, wherein the plurality of dataelements further comprises one or more clinical features related to thediagnosis or treatment of the cancer in the subject selected from thegroup consisting of a stage of the cancer, a histopathological grade ofthe cancer, a therapy administered to the subject, a symptom associatedwith cancer or metastasis thereof, and a comorbidity with the cancer andwherein the (C) applying includes applying the one or more clinicalfeatures to the one or more models.
 4. The method of claim 1, whereinthe plurality of data elements further comprises one or more temporalfeatures related to the diagnosis or treatment of the cancer in thesubject selected from the group consisting of a first temporal elementindicating a duration of time since a diagnosis for the cancer, a secondtemporal element indicating a duration of time since an administrationof a therapy to the subject, a third temporal element indicating aduration of time since an experience of a symptom associated with canceror metastasis thereof, and a fourth temporal element indicating aduration of time since an experience of a comorbidity with the cancerand wherein the (C) applying includes applying the one or more temporalfeatures to the one or more models. 5-7. (canceled)
 8. The method ofclaim 1, wherein the plurality of at least 30 genes comprises at least20 genes selected from the group consisting of the genes listed in Table2.
 9. The method of claim 1, wherein the plurality of at least 30 genesis no more than 250 genes.
 10. (canceled)
 11. (canceled)
 12. The methodof claim 1, wherein the plurality of data elements further comprises asingle-sample gene set enrichment analysis (ssGSEA) score for thetranscriptional profile of the cancer and wherein the (C) applyingincludes applying the single-sample gene set enrichment analysis(ssGSEA) score to the one or more models.
 13. The method of claim 1,wherein the plurality of data elements further comprises a smokingstatus or a menopausal status of the subject and wherein the (C)applying includes applying the smoking status or the menopausal statusto the one or more models.
 14. The method of claim 1, wherein theplurality of data elements further comprises a physical characteristicof the sample of the cancer and wherein the (C) applying includesapplying the physical characteristic to the one or more models.
 15. Themethod of claim 1, wherein the plurality of data elements furthercomprises a mutational status for one or more genes in the sample of thecancer and wherein the (C) applying includes applying the mutationalstatus for the one or more genes to the one or more models.
 16. Themethod of claim 15, wherein the mutational status is for a gene selectedfrom the group consisting of the genes listed in Table
 2. 17. The methodof claim 1, wherein the plurality of data elements further comprises amutational status for one or more genes determined from genomicfragments of a non-cancerous tissue of the subject and wherein the (C)applying includes applying the mutational status for the one or moregenes to the one or more models.
 18. The method of claim 1, wherein theplurality of data elements further comprises a copy number status forone or more genomic regions associated with cancer and wherein the (C)applying includes applying the copy number status for the one or moregenomic regions to the one or more models.
 19. The method of claim 1,wherein the one or more models is a set of models that are collectivelytrained to provide, for each respective tissue in a plurality oftissues, a corresponding set of indications, in the one or moreindications, of whether the cancer will metastasize to the respectivetissue in the subject, wherein the corresponding set of indicationsincludes a corresponding indication for each respective time horizon ina corresponding plurality of time horizons.
 20. The method of claim 19,wherein the set of models comprises, for each respective tissue in theplurality of tissues, a respective subset of models, wherein eachrespective model, in the respective subset of models, is trained toprovide a respective indication of whether the cancer in the subjectwill metastasize to the respective tissue in the subject within arespective time horizon in the corresponding plurality of time horizons.21. The method of claim 19, wherein: the corresponding plurality of timehorizons for each respective tissue in the plurality of tissues is thesame plurality of time horizons; and the set of models comprises, foreach respective time horizon in the plurality of time horizons, arespective model trained to provide, for each respective tissue in theplurality of tissues, a corresponding indication of whether the cancerin the subject will metastasize to the respective tissue in the subjectwithin the respective time horizon.
 22. The method of claim 19, whereinthe set of models comprises, for each respective tissue in the pluralityof tissues, a corresponding model trained to provide, for eachrespective time horizon in the corresponding plurality of time horizons,a corresponding indication of whether the cancer in the subject willmetastasize to the respective tissue in the subject within therespective time horizon.
 23. The method of claim 19, wherein: the set ofmodels comprises a respective model trained to provide, for eachrespective time horizon in a plurality of time horizons, a respectiveindication of whether the cancer will metastasize to any tissue in thesubject; and the plurality of indications of whether the cancer willmetastasize includes, for each respective time horizon in the pluralityof time horizons, a corresponding indication of whether the cancer willmetastasize to any tissue in the subject.
 24. The method of claim 19,wherein the plurality of tissues comprises lymph tissue, liver tissue,and lung tissue.
 25. The method of claim 19, the method furthercomprising: (D) generating a clinical report comprising the one or moreindications of whether the cancer will metastasize; and (E) displayingthe clinical report in a graphical user interface (GUI), wherein the GUIcomprises an anatomical representation of a body and a first affordanceconfigured for switching between respective time horizons in theplurality of time horizons, the displaying comprising: displaying afirst rendering of metastatic predictions comprising, for eachrespective tissue in the plurality of tissues, a corresponding visualrepresentation of the respective indication, in the plurality ofindications, corresponding to whether the cancer in the subject willmetastasize to the respective tissue within a first respective timehorizon, in the corresponding plurality of time horizons, wherein therendering is superposed upon the anatomical representation of the body;and responsive to receiving a user input corresponding to the firstaffordance on the GUI, replacing display of the first rendering ofmetastatic predictions with display of a second rendering of metastaticpredictions comprising, for each respective tissue in the plurality oftissues, a corresponding visual representation of the respectiveindication, in the plurality of indications, corresponding to whetherthe cancer in the subject will metastasize to the respective tissuewithin a second respective time horizon in the corresponding pluralityof time horizons, wherein the rendering is superposed upon theanatomical representation of the body.
 26. The method of claim 1,wherein the one or more models comprises a trained survival function.27. The method of claim 1, the method further comprising: when the oneor more indications of whether the cancer will metastasize satisfies afirst threshold risk for metastasis of the cancer, administering a firsttherapy tailored for treatment of metastatic cancer; and when the one ormore indications of whether the cancer will metastasize does not satisfythe first threshold risk for metastasis of the cancer, administering asecond therapy tailored for treatment of non-metastatic cancer.
 28. Themethod of claim 1, the method further comprising, prior to the (C)applying, training the one or more models using (i) values for thecorresponding plurality of data elements across a plurality of trainingsubjects that have cancer, wherein a portion of the plurality oftraining subjects have metastasized cancer and a portion of the trainingsubjects have cancer that has not metastasized, and wherein thecorresponding plurality of data elements serve as independent variablesin the training and (ii) a corresponding indication for each respectivetraining subject in the plurality of training subjects, of whether therespective training subject's cancer metastasized, wherein theindication serves as a dependent variable in the training therebyobtaining the one or more models that are collectively trained toprovide a respective one or more indications of whether the cancer willmetastasize in a subject. 29-52. (canceled)
 53. A computer system havingone or more processors, and memory storing one or more programs forexecution by the one or more processors, the one or more programscomprising instructions for performing a method for predictingmetastasis of a cancer in a subject, the method comprising: (A)obtaining, in electronic format, a plurality of at least 10,000 sequencereads, wherein the plurality of sequence reads is obtained for aplurality of RNA molecules from a sample of the cancer obtained from thesubject (B) determining from the plurality of at least 10,000 sequencereads, a plurality of data elements for the subject's cancer comprising:a first set of sequence features comprising abundance values for theexpression of a plurality of at least 30 genes in the sample of thecancer obtained from the subject and (C) applying, to the plurality ofdata elements for the subject's cancer comprising the first set ofsequence features comprising abundance values for the expression of theplurality of at least 30 genes, one or more models that are collectivelytrained to provide a respective one or more indications of whether thecancer will metastasize in the subject, thereby predicting whether thecancer will metastasize.
 54. A non-transitory computer readable storagemedium storing one or more programs configured for execution by acomputer, the one or more programs comprising instructions for carryingout a method for predicting metastasis of a cancer in a subject, themethod comprising: (A) obtaining, in electronic format, a plurality ofat least 10,000 sequence reads, wherein the plurality of sequence readsis obtained for a plurality of RNA molecules from a sample of the cancerobtained from the subject (B) determining from the plurality of at least10,000 sequence reads, a plurality of data elements for the subject'scancer comprising: a first set of sequence features comprising abundancevalues for the expression of a plurality of at least 30 genes in thesample of the cancer obtained from the subject; and (C) applying, to theplurality of data elements for the subject's cancer comprising the firstset of sequence features comprising abundance values for the expressionof the plurality of at least 30 genes, one or more models that arecollectively trained to provide a respective one or more indications ofwhether the cancer will metastasize in the subject, thereby predictingwhether the cancer will metastasize.
 55. The method of claim 1, themethod further comprising generating a clinical report comprising theone or more indications of whether the cancer will metastasize.
 56. Themethod of claim 1, wherein the plurality of at least 10,000 sequencereads is at least 100,000 sequence reads.
 57. The method of claim 1,wherein the plurality of at least 10,000 sequence reads is at least1,000,000 sequence reads.
 58. The method of claim 1, wherein theabundance values for the expression of a plurality of at least 30 genesare relative abundance values.